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Many  image  analysis  tasks  require  the  construction  of  a 
boundary  representation  as  a  means  of  partitioning  an  image. 
This  paper  develops  a  parallel  relaxation  algorithm  for  updating 
initial  heuristic  estimates  of  the  likelihood  of  edges  so  that 
continuous  boundaries  are  formed.  Bayesian  probability  theory  is 
used  to  analyze  the  probability  updating  of  a  single  edge  based 
upon  the  joint  probabilities  of  the  edges  in  its  local 
surrounding  context.  The  relationships  between  edges*  sometimes 
referred  to  as  compatibi lity  coefficients  in  relaxation 
algorithms*  are  embodied  as  conditional  probabilities  between  the 
central  edge  and  the  context  of  edges.  The  set  of  conditional 
probabilities  are  theoretically  derived  from  a  model  of  desired 
line  drawings  that  satisfy  basic  notions  of  boundary  continuity. 
The  local  updating  function  attempts  to  drive  the  likelihood  of 
each  central  edge  into  consistency  with  the  surrounding  context. 

.Experiments  involving  the  iterative  parallel  application  of 
this  non-linear  Bayesian  updating  function  to  all  edge 
probabilities  demonstrates  serious  problems  in  the  formulation. 
A  variety  of  heuristic  modifications*  guided  by  theoretical 
considerations*  are  examined  empirically.  The  final  formulation 
is  an  algorithm  that  performs  in  an  effective  manner  on  several 
very  complex  images. 
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I.  INTRODUCTION 


Many  image  processing  tasks  require  construction  of  a 
two-dimensional  boundary  representation  of  an  image.  The  sensory 
data  is  usually  presented  as  an  array  of  feature  values.  Each 
array  element*  referred  to  as  a  "pixel”  (for  "picture  element" >* 
is  characteri zed  by  its  spatial  location  within  the  array  and  by 
the  values  of  the  sensory  features  at  that  point.  For  our 
purposes  it  will  be  assumed  that  a  pixel  value  represents  a 
measure  of  the  amount  of  light  which  lands  on  a  small  area  of 
some  imaging  device.  Typically*  local  edge  operators 
CDAV75*  R0S76*  RIS773  are  used  to  provide  estimates  of  the  local 
strength  of  edges  throughout  the  image*  but  the  results  produced 
are  almost  always  unreliable.  The  raw  edge  information  requires 
far  more  structural  organization  in  order  to  be  useful. 

In  this  paper  we  seek  methods  for  partitioning  the  image  by 
focussing  on  local  edges  between  pixels  and  on  the  means  by  which 
they  may  be  organized  into  continuous  boundaries.  The  primitive 
element  of  a  boundary  is  an  edge  element.  A  set  of  connected 
edge  elements*  located  relative  to  the  array  of  sensory  data  from 
which  it  was  derived*  forms  a  boundary.  We  sometimes  will  use 
the  term  "line"  synonymously  with  "boundary"  because  the  result 
of  this  form  of  segmentation  is  a  "line  drawing".  The  approach 
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described  here  addresses  the  problem  of  how  to  make  use  of  local 
contextual  information  to  reduce  local  edge  ambiguity  while 
producing  a  global  organization  of  continuous  boundaries.  The 
initial  array  represents  sensory  data  while  the  final  array 
represents  a  boundary  continuity  interpretation  of  that  data  — 
or  controlled  hallucination  in  imagining  a  line  drawing 
consistent  with  the  sensory  data! 

1.  1  Rtvigye  S±  Related  Literature 

Methods  vary  for  dealing  with  the  uncertainty  which  remains 
after  local  edge  operators  have  been  applied.  Marr  [MARR763  uses 
a  set  of  edge  detectors  of  varying  size  to  determine  the 
appropriate  width  of  an  edge  to  be  asserted  in  his  primal  sketch. 
Ehrich  CEHR79a3  has  devised  an  edge  detector  whose  output  is  not 
just  a  local  edge  assertion*  but  a  data  structure  containing 
alternative  interpretations  of  edge  cross-sections. 

Sequential  edge  tracking  methods  have  benefited  from  current 
techniques  for  dealing  with  uncertain  or  noisy  data.  Montonari 
has  embedded  the  properties  of  a  curve  in  a  figure  of  merit 
CMON713.  A  dynamic  programming  technique  is  used  to  determine 
the  optimal  curve  with  respect  to  the  given  figure  of  merit. 
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Martel li  CMART763  hat  reformulated  this  Method  at  a  graph  search 
problem  which  finds  a  contour  using  an  optimizing  heuristic 
search.  Cooper  models  the  sequence  of  edge  elements  in  a  contour 
as  a  Markov  process  and  uses  maximum  likelihood  estimation  to 
generate  the  contour  CC0Q793.  Fischler  et  al.  CFIS793  apply  the 
optimization  paradigm  to  the  specific  problem  of  detecting  roads 
and  linear  structures  in  aerial  imagery.  They  also  introduce 
mechanisms  for  combining  local  evidence  and  constraints  into  the 
global  optimization  process. 

Recently  there  has  been  a  large  amount  of  research  in  local 
iterative  processes  called  relaxation  labelling  problems 
CR0S76*  ZUC783.  The  fundamental  idea  of  relaxation  labelling  is 
to  iteratively  update  the  likelihood  of  local  interpretations 
based  upon  expected  local  relationships  between  these 
interpretations.  At  each  iteration  the  influence  of  a  context  of 
nodes  will  spread  indirectly  through  the  influence  of  the  nodes 
which  it  affected.  Ideally*  the  entire  system  will  globally 
organize  to  be  locally  consistent*  and  possibly  even  display 
global  characteristics  which  would  have  been  very  difficult  to 
achieve  as  a  global  function  specified  directly  in  terms  of  all 


the  data. 
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Current  approaches  to  relaxation  labelling  have  employed 
heuristic  mechanisms  for  updating  likelihoods.  Much  of  this  work 
is  based  on  the  formal ixation  provided  in  CR0S763*  referred  to  as 
the  non-linear  probabilistic  rule.  In  these  systems  local 
contextual  information  is  embodied  in  compatibility  coefficients 
which  parameterize  the  mutual  effects  of  adjacent  labellings. 
Some  possible  interpretations  of  these  coefficients  are  presented 
in  CPEL783  including  measures  of  correlations  and  mutual 
information.  Nevertheless!  these  parameters  are  still  heuristic. 
Zucker  has  argued  for  an  interpretation  of  these  compatibility 
coefficients  as  measures  of  dependency  EZUC783.  He  has 
reformulated  the  relaxation  updating  rule  mentioned  above  into 
one  involving  conditional  probabilities.  While  intended  as  an 
analysis  of  non-linear  probabilistic  updating!  this  formulation 
is  similar  to  the  most  successful  of  the  updating  rules  we 
present  here.  Peleg  has  also  developed  a  probabilistic 
relaxation  labelling  scheme  which  uses  conditional  probabilities 
CPEL793.  This  work  shares  our  intent  of  providing  an  analytical 
basis  for  relaxation. 
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1.2  Comolexitu  of  Segmentation 

With  nlativdy  unconstrained  natural  images*  any  approach 
to  segmentation  which  raliaa  entirely  on  the  sensory  data*  will 
he  prone  to  error.  The  complexity  of  the  data  is  a  function  of  a 
variety  of  factors  including  the  presence  of  highly  textured 
objects*  shadows  and  highlights  on  smooth  or  irregular  surfaces* 
variation  in  surface  reflectance*  varied  and  uncontrolled 
lighting*  and  noise  introduced  in  the  digitization  process.  Few 
objects  or  surfaces  in  an  image  can  be  expected  to  exhibit 
uniform  visual  features.  In  the  images  considered  here*  such  as 
the  ones  shown  in  Figure  1*  it  is  very  unclear  exactly  where  some 
of  the  boundaries  should  be  placed  in  relation  to  the  local  array 
of  feature  values. 

t 

Even  human  hand-drawn  segmentations  are  inevitably  prone  to 
errors  and  tend  to  reflect  implicit  biases  and  explicit  goals  of 
the  human  perceiving  the  image.  In  many  instances  the  boundaries 
would  be  conjectured  based  on  prior  expectations  in  the  form  of 
knowledge  of  objects  and  their  shape*  shadow  effects*  perspective 
and  occlusion  cues*  etc.  CHAN78b3.  In  short*  it  is  generally 
accepted  that  a  truly  accurate  segmentation  is  a  function  of  the 
goals  of  the  system  and  requires  the  application  of  "high-level" 
knowledge  which  is  not  directly  available  as  measurable  features 


Figure  1  continued 
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of  light.  A  discussion  of  tho  problras  of  ond  Approaches  to 
evaluation  appears  in  CNAQ793. 


I. 3  Boundaru  FgTflitogn 

The  edge/boundary  process  presented  in  this  paper  cannot  be 
expected  to  provide  a  perfect  boundary  segmentation  of  the  image 
because  no  such  segmentation  exists.  Rather  we  look  for 
processes  which  can  provide  boundaries  to  a  good  first 
approximation*  where  the  feature  data  exhibits  relatively  large 
and  spatially  consistent  discontinuities.  We  will  utilise  the 
notion  of  variation  in  sensitivity  of  the  boundary  formation 
process  so  that  it  can  be  used  to  extract  weaker  and  more 
ambiguous  boundaries  if  so  desired.  In  particular  we  do  not 
expect  this  edge/boundary  algorithm  to  distinguish  between 
texture  elements  and  the  boundary  of  textured  objects.  Thus* 
strong  texture  will  lead  to  boundaries  around  texture  elements 
and  possibly  a  high  density  of  edges  within  a  textured  region. 

Our  goals  in  the  segmentation  of  an  image  can  be  stated  two 

ways: 


j 
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1.  to  partition  tha  image  into  (disjoint)  sets  of  pixels, 
called  regions.  which  have  relatival u  invariant  visual 
features. 

2.  to  form  continuous  boundaries  by  placing  edges  between 
pixels  which  have  relativel u  large  differences  in 
feature  values. 

When  the  visual  data  is  very  simple  <e.  g.  .  a  cartoon  image  with 
closed  boundaries  and  without  textural  variation),  these  two 
approaches  are  equivalent.  However.  when  the  data  is  more 
complicated  and  has  a  high  degree  of  ambiguity  in  certain  places, 
then  these  approaches  almost  certainly  will  produce  different 
segmentations  and  will  use  different  analyses  of  the  data  to 
achieve  the  goal. 

The  edge/boundary  process  developed  in  this  paper  is  a 
probabilistic  edge  labelling  relaxation  algorithm  based  on  a 
simplified  edge  representation  and  a  notion  of  boundary 
continuation  in  digital  images.  In  this  algorithm,  information 
in  the  context  of  a  given  edge  element  is  used  in  an  attempt  to 
drive  the  central  edge  into  consistency  with  the  surrounding 
context.  The  edge/boundary  process  is  initially  formulated  in 
Bayesian  probability  theory.  A  model  of  desired  line  drawings  is 
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used  to  determine  theoretical  estimates  of  the  conditional 
probabilities  relating  an  edge  to  the  joint  probability  of  edges 
in  the  context. 

Some  of  the  data  used  in  this  paper  is  shown  in  Figure  1. 
In  all  cases  the  feature  used  is  intensity*  which  was  derived 
from  the  original  tri-stimulus  color  data.  Our  full  images  are 
actually  subimages  which  are  extracted  from  data  whose  original 
resolution  was  256x256  pixels  with  at  least  6  bits/color.  In 
order  to  give  a. sense  of  our  goals  and  methodology*  the  results 
produced  by  the  edge/boundary  process  which  is  developed 
incrementally  in  the  rest  of  the  paper  are  previewed  in  Figures  2 
and  3.  The  initial  probability  assigned  to  an  edge  is  a  function 
of  both  the  local  edge  strength  determined  by  application  of  a 
1x2  edge  mask  and  the  strength  of  other  edges  in  the  immediate 
vicinity*  this  initial  assignment  is  shown  in  Figure  2. 
Beginning  with  this  initial  data*  the  edge/boundary  process 
updates  every  edge  in  the  image  simultaneously  (in  parallel)  on 
the  basis  of  data  contained  in  a  small  neighborhood  around  each 
edge.  This  updating  process  is  repeated  until  most  of  the  edges 
do  not  change  from  one  iteration  to  the  next.  The  results  shown 
in  Figure  3<a)-(d)  appear  to  be  quite  reasonable  for  most  areas 
of  the  image. 


Figure  2..  Results  of  Edge  Relaxation  Por  Boundary  Continuity. 

Results  from  the  final  Pormulation  of  tha  adga 
relaxation  process  developed  in  Section  IV.  In  most 
cases  the  boundaries  fit  a  very  reasonable 
representation  of  the  underlying  data  and  with  minor 
exceptions  correspond  quite  well  to  visually  apparent 
boundaries. 
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II.  A  BAYESIAN  VIEW  OF  EDOE  CONTINUITY 

ii.  i  uul  si.  EJm 

Our  representation  of  tdgt  information  is  highly  structured 
with  each  edge  having  an  unambiguous  location  and  orientation. 
In  particular*  a  horizontal  odgo  may  bo  located  between  any  pair 
of  vertically  adjacent  pixels*  a  vertical  edge  may  be  located 
between  any  pair  of  horizontally  adjacent  pixels  (refer  to  Figure 
4).  Edge  presence  at  each  horizontal  and  vertical  edge  location 
is  encoded  with  some  numeric  measure  of  edge  strength  or 
likelihood.  During  the  boundary  formation  process  this 
representation  avoids  the  ambiguities  in  precise  location  and 
orientation  of  edges  that  would  occur  if  edge  information 
relative  to  pixel  locations  were  not  encoded.  The  issues 
Involved  in  this  choice  of  this  representation  are  discussed  in 
CHAN7Ba*  PRA803. 


Our  approach  to  boundary  formation  makes  use  of  two  basic 
assumptions: 


1.  the  contrast  assumption:  the  likelihood  of  a  local  edge 
is  directly  proportional  to  its  contrast* 
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represents  a  pixel  location 
represents  an  edge  location 
represents  vertex  locations 


Figure  4.  The  Representation  of  Edges  in  VISIONS.  This  interpixel 
representation  of  horizontal  and  vertical  edges  between 
adjacent  pixels  allows  the  placement  and  orientation  of 
edges  to  be  precise.  The  location  of  possible  boundary 
junctions  and  terminations  follows  from  the  edge  positions. 


□  igigig 
□jgjgjg  E 
gigigig 

□  onin 
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and  2.  th*  continuity  assumption:  boundaries  in  images  are 

continuous  almost  everywhere;  that  is*  every  "real" 
edge  element  is  joined  to  one  or  more  adjacent  edge 
elements  almost  everywhere. 

The  contrast  assumption  allows  us  to  compute  the  likelihood  of  a 
local  edge  based  solely  on  its  contrast*  while  the  continuity 
assumption  provides  a  mechanism  for  propagating  information  from 
a  "context”  to  the  edge  in  question. 

Our  goal  will  be  to  define  local*  parallel  mechanisms  by 
which  initial  probabilities  of  edges  can  be  updated  so  that 
uncertainty  of  edge  presence  is  reduced  while  edges  are 
aggregated  into  continuous  boundaries.  The  means  by  which  this 
will  take  place  is  based  on  the  updating  of  an  edge  probability 
via  a  Bayesian  view  of  the  joint  probability  of  edges  in  the 
local  context  surrounding  that  edge. 

ii. 2  Hut  Probability  af.  an  £ Isls. 

As  we  have  already  noted*  the  image  data  to  be  analyzed  is 
not  very  well-behaved.  It  is  extremely  difficult  to  make  an 
absolute  decision  about  the  presence  or  absence  of  a  particular 
edge  element  based  purely  on  local  information  around  that  edge. 
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Consequently.  it  is  desirable  to  avoid  such  absolute 
classification  during  the  early  stages  of  visual  organization  of 
information  CMARR76. EHR793.  otherwise#  one  must  face  the 
computational  burdens  of  undoing  erroneous  decisions.  One 
plausible  strategy  would  be  to  delay  any  decision  at  this  point 
and  to  carry  forward  all  possible  decisions  so  that  contextual 
data  can  be  used  in  later  stages  to  select  the  best  choice.  All 
possible  local  decisions  —  or  labellings  —  at  a  single  location 
are  retained.  The  approach  advocated  here  has  some  of  this 
flavor  mixed  with  the  notion  of  spreading  of  contextual 
information  via  the  parallel  updating  of  edge  likelihoods. 

In  our  representation  there  are  only  two  decisions  —  or 
labellings  —  at  a  single  location:  "edge”  and  "no-edge".  A 
probability  or  degree  of  confidence  will  be  associated  with  each 
label  at  each  edge  location.  The  term  probability,  rather  than 
some  heuristic  notion  of  "confidence",  will  be  used  here  because 
we  will  be  guided  by  a  Bayesian  view  of  edge  presence.  Only  a 
single  value  is  required  to  store  the  probability  of  two  labels 
at  each  edge  location  if  the  label  set  is  assumed  to  be  mutually 
exclusive  and  exhaustive. 

Uncertainty  in  the  existence  of  an  edge  can  be  expressed  in 
various  ways.  We  will  denote  the  probability  that  some  edge  E  is 
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present  (true)  or  absent  (false)  with  the  random  variable  E» 
which  can  take  on  values  T  or  F .  respectively.  Then,  we  can 
represent  the  probability  of  edge  presence  as  P(E“T)  and  of  edge 
absence  as  P(E»F)  *  1  -  P(E*T). 

Via  the  contrast  assumption,  the  probability  of  an  edge  is 
related  to  the  difference  in  feature  values  of  the  pixels  in  the 
vicinity  of  that  edge.  Let  us  leave  aside.  until  the  next 
section.  the  problems  of  gradual  feature  changes  over  a  sequence 
of  pixels,  i.  e.  >  wide  non-zero  feature  gradients.  Thus.  edge 
contrast  varies  over  the  same  range  as  the  feature  values  and  can 
be  scaled  into  a  zero-one  range  to  represent  edge  probability. 

In  this  paper  the  conversion  from  edge  contrast  values  to 
edge  probabilities  is  performed  by  normalizing  the  maximum 
feature  contrast  over  a  large  local  neighborhood  of  an  edge.  The 
strongest  edge  in  each  neighborhood  can  have  a  probability  of  one 
and  edges  with  zero  feature  difference  will  be  mapped  into 
probability  zero.  For  each  edge  location  in  the  image,  we 
extract  the  maximum  and  minimum  feature  values.  say  Fmax  and 
Fmin.  within  a  local  11x11  window  centered  on  the  edge  under 
consideration.  The  edge  probability  is  then  determined  by 


P(edge)  -  E’ |  [^(F^  -  P^)]} 


(1) 
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nktri  E '  is  tha  local  odgo  obtainod  by  diffarancing  tha  adjacant 
pixal  valuaa  (a.  g.  *  a  1x2  diffaranca  aatk)<  and  K  is  a  local 
scaling  factor  which  allows  tha  contrast  valua  which  is 
considarad  to  hava  a  probability  of  ona  to  ba  variad. 

Thara  ara  two  paramatars  that  hava  baan  hauristically  sat  at 
intuitivaly  raasonabla*  but  arbitrary*  constant  valuas. 
Variation  of  tha  sisa  of  tha  naighborhood  would  changa  tha 
locality  of  influanca  in  satting  initial  probabilitias*  whila 
variation  of  tha  scaling  paramatar  K  wight  sarva  as  an  adga 
sansitivity  control.  Tha  affact  of  changing  thasa  parawatars  is 
not  axplorad  in  this  papar.  Tha  contrast  naighborhood  has  baan 
fixad  at  11x11  and  K  has  baan  sot  to  .5  for  all  rasults  prasantad 
in  this  papar.  Figura  9  illustratas  tha  convarsion  of  adga 
contrast  to  adga  probability  for  a  sub  imago  of  Figura  1(a). 
Figura  9(a)  raprasants  tha  output  of  tha  1x2  adga  wash  in  which 
brightnass  ancodas  adga  strangth.  Nota  that  thara  ara  wany  waak 
adgas  which  ara  not  avidant  in  tha  figura  as  it  has  baan 
raproducad  hara.  Figura  9(b)  shows  tha  adga  probabilitias 
obtainod  by  applying  aquation  1  at  avary  horixontal  and  vortical 
adga  location.  Nota  that  tha  boundary  on  tha  laft  sida  of  tha 
bush  is  wora  cowplata  and  strongar.  Bacausa  tha  normalisation  is 
parforwad  locally  ovar  an  11x11  window*  non-linaar  scaling  way 
occur.  A  ralatad  affact*  which  can  ba  saan  in  tha  wall  araa 
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ure  Conversion  of  Edge  Strength  to  Edge  Probability.  (a) 

Result  from  1x2  edge  mask  applied  to  Figure  1(c). 
Although  it  is  not  evident  in  the  picture>  there  are 
a  large  number  of  very  weak  edges  throughout  the 
image.  <b)  Edge  probabilities  obtained  by 

normalizing  edge  strength  over  an  11x11  local  window 
centered  on  each  edge.  Compare  the  left  side  of  the 
bush  in  the  two  images. 
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above  tha  roof  line,  is  the  relative  increase  in  the  strength  of 
weak  edges  in  areas  where  there  are  only  weak  edges  (and  low 
maximum  contrast). 


II. 3  gradient  Collection 

One  objection  which  may  be  raised  against  the  use  of  a  1x2 
edge  operator  is  related  to  those  situations  in  which  a  boundary 
is  actually  represented  by  a  non-zero  feature  gradient  that  is 
wide  relative  to  the  mask  size.  The  highlight  evident  around  the 
rim  of  the  bush  in  Figure  6(a)  is  a  function  of  the  light  source, 
the  surface  properties  of  the  bush  and  background,  and  the 
relative  orientations  of  the  camera,  the  source,  and  the  relevant 
surfaces.  It  might  be  argued  that  a  sophisticated  model  of 
intensity  changes  is  necessary.  one  perhaps  in  which  the 
relationships  between  the  geometry  of  the  imaging  process  and  the 
resulting  image  gradient  are  taken  into  account  CHOR77. BAR7B3. 
Although  this  approach  may  prove  to  be  very  effective,  it  has 
only  been  explored  in  artificially  constructed  domains. 
Extensions  of  the  approach  to  an  analysis  of  the  physics  and 
geometry  of  images  containing  foliage  (for  example)  do  not  appear 
to  be  immediately  feasible.  We  will  view  the  total  change  in 
feature  value  across  the  width  of  a  gradient  as  a  single  value  to 
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be  used  in  determining  the  likelihood  of  an  edge  somewhere  within 
the  gradient.  This  approach  is  described  in  more  detail  below 
and  it  seems  to  work  reasonably  well  in  our  test  images. 

In  many  cases  a  gradient  edge  is  produced  whenever  an 
environmental  edge  is  not  parallel  to  one  of  the  axes  of 
digiti2ation  (Figure  6(a*b)).  The  degree  of  “smearing"  is  a 
function  of  the  slope  of  the  edge  and  the  digitizer 
characteristics  (size  of  the  active  sensor*  sampling  rate*  etc.). 
An  example  of  this  effect  is  clearly  seen  in  the  sloping  roof 
edges  of  Figure  7(a)  where  the  actual  roof  edge  in  the  original 
photograph  is  a  sharp  edge. 

In  cases  such  as  these*  the  mask  will  respond  to  only  a 
portion  of  the  full  contrast  which  is  distributed  over  several 
pixels.  This  has  led  to  approaches  in  which  a  hierarchical  set 
of  increasingly  larger  masks  is  employed 
CR0S71* HAN74* MARR76.  HAN803.  The  approach  suggested  here  — 
referred  to  as  "gradient  collection”  —  is  to  extract  the  total 
contrast  of  a  boundary  by  collecting  the  local  horizontal  and 
vertical  components  of  the  gradient  (as  obtained  from  the  1x2 
mask)  and  to  place  the  resultant  value  at  a  representati ve  edge 
location  CHAN78a3.  It  is  essentially  an  application  of  the 
contrast  assumption  to  edges  where  the  non-zero  gradient  is 
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spatially  distributed.  The  probability  of  the  resulting  edge  may 
then  be  determined  using  the  method  presented  in  the  preceding 
section.  This  is  discussed  more  fully  in  Section  IV.  Note  that 
in  effect  this  strategy  is  an  avoidance  of  fixed  mask  sizes  and 
geometriesi  the  effective  mask  size  is  determined  dynamically  as 
a  function  of  the  local  data. 


Our  gradient 

collection  process 

involves 

selection  of 

an 

edge  location 

with  in 

the  width 

of  the 

gradient  as 

the 

representative  canonical 

location  of 

the  entire 

gradient  edge* 

as 

well  as  the  specification  of  the  likelihood  to  be  associated  with 
the  gradient  edge  as  a  function  of  the  total  contrast.  In 
general*  edges  will  have  to  be  placed  at  several  representative 
locations  to  minimize  misalignment  of  adjacent  edges.  This 
misalignment  can  occur  if  adjacent  views  of  the  gradient  differ 
due  to  noise  or  systematic  changes  in  the  extent  of  the  gradient. 

Gradient  collection  is  performed  during  the  assignment  of 
initial  edge  probabilities.  All  horizontal  runs  of  consecutive 
vertical  edges  and  vertical  runs  of  consecutive  horizontal  edges 
across  a  non-zero  gradient  and  with  the  same  direction  of 

m 

contrast  ore  considered  as  gradient  edges.  As'  illustrated  in 
Figure  b,  three  multiplicative  factors  determine  the  final 
probability  assignment  to  each  constituent  edge  in  the  gradient: 
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1.  the  location  factor  -  the  canter  of  gravity  <c.  g.  )  of 
tha  edge  strengths  in  a  gradient  is  used  as  the  point 
about  which  an  inverse  distance  weighting  is  computed# 
the  location  factor  will  be  assigned  a  value  of  1.0  at 
the  c. g .  #  a  value  of  0  for  edges  at  a  distance  of  4 
pixels  or  greater  from  the  c. g. #  and  linear 
interpolation  for  all  edges  at  a  distance  from  the  c.  g. 
between  0  and  4$ 

2.  the  relative  edge  strength  factor  -  all  edges  in  a 
gradient  are  divided  by  the  maximum  edge  so  that  the 
places  of  strongest  contrast  change  will  receive  a 
greater  weighting)  and 

3.  the  collected  edge  strength  factor  -  the  collected 
strength  of  the  gradient  Ci.e.  *  the  sum  of  all 
constituent  edges)  is  assigned  a  probability  by 
normalizing  by  the  maximum  local  contrast#  which  is 
scaled  by  a  factor  K  and  bound  by  a  maximum  value  of  1# 
as  in  equation  (1). 


These  three  factors  when  multiplied  give  the  final  edge 
probabilities  along  a  gradient.  It  is  easy  to  verify  that  this 
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Mtkei  has  uvtra)  desirable  proparti®*.  Th®  inv®r»®  distance 
weighting  about  th®  cantor  of  gravity  focus*®*  ®vid®ntial  weight 
on  a  few  liholg  candidat®*.  Noraali cation  of  individual  ®dg® 
•trongth*  Maintain*  soa*  inforaation  about  th®  relative  ®dg® 
strength  in  a  gradient*  while  th®  collected  ®dg®  strength*  ov®r 
th®  gradient  allow*  th*  full  contrast  to  influonc®  th®  final 
probability  of  odgo  prosonco. 

Figure  7  illustrates  th*  application  of  the  gradient 
collection  procoss  to  actual  data.  For  illustrative  purposes*  a 
saall  portion  of  th®  roof  trial  (Figure  7a»b)  was  chosen  bveaus® 
of  th*  horizontal  and  vortical  gradients  both  above  and  below  th® 
tria  area.  Figure  7(c)  shows  th*  result  of  applying  a  1x2  ®dg® 
•ask  to  th*  intensity  data;  in  this  figur*  ®dg®  intensity 
encodes  odgo  strength.  Th*  picture  clearly  shows  th® 
aiultipl icity  of  edges  caused  by  the  gradients.  Figur*  7(d)  is 
th*  results  obtained  froai  the  gradient  collection  process 
described  above;  here*  brightness  encodes  edge  probability. 
Without  changes  in  the  display  scaling  th*  effects  of  th* 
gradient  collection  process  are  hard  to  see*  but  there  is  aor* 
consistency  in  the  presence  of  on*  or  two  strong  edge  locations 
down  th*  entire  length  of  the  boundary.  Otherwise  any  weak  spot* 
particularly  in  th*  vertical  position*  cause  th*  boundary  to 
brook  up  during  relocation.  Zt  is  difficult  to  evaluate  the 
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ire  7.  Application  of  the  Gradient  Collection  Process.  (a) 
Intensity  image  of  houre  showing  location  of  roof 
trim  subimage.  <b)  SxS  section  of  roof;  note  wide 
non-zero  gradients  both  shove  and  below  the  roof 
trim.  <c)  Results  of  1x1?  edge  mask.  (d> 

Gradient-collected  edge  probab i 1 i t i es;  edge 

brightness  is  a  function  of  probability  (0  =  black,  1 
■  white).  (e-i)  Iterations  1.  2 ,  3,  10,  and  20 

(respectively )  of  the  edge  relaxation  process 
developed  in  the  paper  based  on  the  initial 

gradient-collected  edge  pr oba b  i  1  i t i es;  the  final 

result  is  a  reasonable  boundary  representation  of  the 
roof  trim  subimage. 


effectiveness  of  the  collection  in  isolation  from  the  relaxation 


process  which  will  make  use  of  the  data.  Figure  7<e-i)  shows  the 
application  of  the  edge  relaxation  process  developed  in  the 
remainder  of  the  paper.  It  is  apparent  that  by  20  iterations 
(Figure  7i )  the  boundary  obtained  is  a  reasonable  representation 
of  the  visual  boundary  in  7(b).  Note  that  the  boundary  in  7(i) 
is  composed  only  of  horizontal  and  vertical  edge  elements.  As  a 
result*  there  is  no  information  locally  available  about  global 
characteristics  of  the  boundary*  such  as  the  slope  of  the  roof 
trim.  Zn  our  system*  the  final  boundary  results  shown  here  are 
processed  further  and  relevant  features  extracted  CY0R803. 

While  we  have  presented  a  totally  heuristic  strategy  for 
assigning  initial  probabilities:  the  factors  are  all  relevant  to 
edge  placement  and  provide  intuitively  reasonable  values.  Better 
models  of  gradient  collection*  perhaps  guided  by  psychological 
considerations  in  boundary  perception*  are  called  for. 


1 1.  4  &  Bauesian  View  of  the  Local  Context  of  an  Edoe 

The  simplest  view  of  the  boundary  formation  process  involves 
the  use  of  the  information  contained  in  the  context  of  a  single 
edge  to  affect  the  degree  of  belief  in  the  existence  of  that  edge 
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as  part  of  a  continuous  boundary.  Given  the  hori2ontal/vertical 
edge  representation  that  we  employ*  the  smallest  neighborhood 
which  allows  contextual  information  to  propagate  by  means  of 
boundary  continuity  is  shown  in  Figure  3.  Such  a  neighborhood 
must  include  the  three  edge  locations  to  either  side  of  a  given 
edge  where  continuation  of  that  edge  can  occur.  Shortly*  we  will 
expand  the  local  context  to  include  the  adjacent  parallel  edges 
so  that  competing  alternate  locations  for  the  placement  of  wider 
gradient  changes  can  be  handled  in  a  local  parallel  organization 
of  the  information.  However*  let  us  first  examine  the  simpler 
context  of  only  the  six  potential  continuation  edges  around  any 
given  horizontal  or  vertical  edge. 

Given  the  conditions  of  uncertainty  in  the  presence  of  most 
edges*  the  goal  is  to  use  the  neighborhood  context  to  improve  our 
estimate  of  the  probability  of  the  central  edge.  It  is  natural 
to  turn  to  Bayesian  probability  theory  for  the  definition  of  the 
relationship  of  the  central  edge  to  the  neighborhood  context. 
Letting  random  variable  H  be  associated  with  the  central  edge* 
the  probability  that  central  edge  H  is  present  will  be  P<H*T>  and 
can  be  determined  as  the  sum  of  the  joint  probability  of  each  of 
the  mutually  exclusive  2**6  *  64  ways  that  H  can  occur  within  the 
context  of  the  surrounding  edges: 


Figure  8.  Context  of  Edge  H.  The  smallest  neighborhood  which  allows 
contextual  information  to  propagate  to  H  must  include  the 
possible  boundary  continuations  to  the  left  (edges  El,  E2, 
and  E3)  and  to  the  right  (edges  E4,  E5,  and  Eg) •  Note  that 
+  represents  vertex  locations  where  horizontal  and  vertical 
edges  can  meet. 
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P(H-T)  -  P(H-T,  Ej-T,  E2-T,...,E6-T) 

+  P(H*T,  E^*F,  E2“T t • • • »Eg“T) 

+  •  •  • 

+  P(H*T,  E2*Fj i • • 

This  can  b»  shortanad  to: 

P(H-T)  -  l  •••  I  P(H-T,  E,,  E2,...,E6) 

E1  E6 

where  th*  summation  involving  each  Ei«  i  =  1*...*6»  is  enumerated 
over  Ei  *  <T*F>.  In  effect  the  likelihood  of  the  central  edge 
given  a  context  of  six  local  edges*  where  each  edge  Ei  in  the 
context  may  be  uncertain*  is  a  linear  interpolation  of  the 
likelihood  of  the  64  six-tuples  for  which  there  is  certainty 
about  the  joint  presence  and  absence  of  each  edge. 

Using  the  standard  definition  of  conditional  probability  the 
joint  probability  of  the  central  edge  and  context  can  be 
decomposed; 

P(H-T)  -  l  ...  £  P(H-T|E1,...iE6)  P(Er...,E6)  (3) 

E1  E6 

The  conditional  probability  of  the  central  edge*  given  a  state  of 
certainty  in  the  context  of  six  edges*  will  embody  the  semantics 
of  boundary  continuity. 
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II.  5  Equivalence  Classes  a£  Edge  Contests 

By  taking  into  account  natural  symmetries  in  the  local 
neighborhood*  the  64  cases  which  must  be  explicitly  considered 
can  be  significantly  reduced  CHAN78a*  PRA803.  To  accomplish  this, 
a  labelling  convention  for  the  edge  patterns  that  can  appear  in 
the  left  and  right  contexts  of  the  central  edge  is  required. 
With  the  generality  of  our  continuity  assumption*  it  is  not 
necessary  to  take  into  account  the  orientation  of  the  extension 
of  the  central  edge  into  the  context.  Note  that  this  would  be 
important  if  straight  boundaries  <or  boundaries  with  some  other 
parameterized  2D  curvature)  were  to  be  extracted*  but  in  this 
paper  no  constraints  on  boundary  curvature  other  than  continuity 
are  considered. 

Extensions  of  the  central  edge  into  the  context  can  best  be 
categorized  by  labelling  the  two  vertices  of  any  given  edge.  A 
vertex  label  will  be  a  function  of  the  number  of  edges  present  in 
the  three  edge  locations  which  can  serve  as  the  continuation  of  a 
boundary  which  passes  through  the  central  edge.  Thus*  we  will 
represent  the  four  possible  equivalence  classes  of  vertices  as 
VO*  VI.  V2*  and  V3>  where  Vi  represents  the  case  where  there  are 
exactly  i  edges  present  at  a  vertex  in  the  context.  Thus*  the 
three  possible  single  edge  continuations  at  a  vertex  of  the 
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central  edge  are  treated  as  an  equivalence  class*  as  shown  in 
Figure  9. 

The  notation  naturally  extends  to  the  full  representation  of 
our  context  of  six  edges  by  considering  jointly  the  vertices  to 
the  right  and  left.  Thus*  Vij  will  be  used  to  represent  a 
context  with  a  vertex  labelled  Vi  to  the  left  and  a  vertex 
labelled  Vj  to  the  right.  For  vertical  central  edges  Vij  will 
represent  top  and  bottom  vertices.  Now*  any  of  our  64  six-tuples 
of  edges  which  are  present  or  absent  can  be  represented  by  one  of 
the  16  equivalence  classes  Vij*  O  £  i*  j  £  3.  Thus*  the 
equivalence  classes  produced  by  symmetry  of  the  number  of  edges 
reduced  the  64  contexts  to  16  contexts.  By  the  additional 
left-right  symmetry  of  vertices  <which  does  not,  of  course, 
affect  continuity  considerations ) *  we  can  further  reduce  the  set 
of  equivalence  classes  to  the  ten  Vij,  0  <  i  £  j  <  3.  Figure  10 
is  an  enumeration  of  these  ten  classes  in  which  an  unfilled  bar 
represents  the  central  edge  H*  a  solid  line  represents  an  edge  in 
the  context  which  is  present*  and  a  dotted  line  represents  an 
edge  location  where  an  edge  is  absent. 

In  summary*  there  has  been  a  significant  reduction  in  the 
number  of  local  contexts  which  must  be  considered  for  boundary 
continuation.  It  was  achieved  by  assuming  left-right  symmetry 
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Class 
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vo  +  cm 


VI 


V2 


V3 


H 


Figure  9.  Equivalence  Classes  of  Vertex  Types.  Each  type  represents 
an  equivalence  class  of  line  continuations  in  which  the 
number  of  possible  continuations  is  fixed.  Note  that  by 
symmetry  a  similar  labelling  convention  holds  when  the 
central  edge  is  to  the  left  of  the  vertex. 


36 


i  • 

i  t 

(  t 


•  i 

i  i 

voo 


VO  3 


VI 2  VI 3 


Ij 

r. 


1 1 

C3 

I  I 


I  I 
I  I 


Fissure  10.  Ten  equivalence  classes  of  edge  contexts  based  on  the  number  of 
edges  in  each  vertex  and  subject  to  left-right  symmetry. 
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and  counting  only  the  number  of  edges  within  each  half  of  the  six 
edge  context  while  disregarding  the  actual  edge  orientation  and 
edge  placement.  As  we  have  pointed  out.  orientation  can  be  very 
useful,  and  it  is  employed  in  our  system  at  a  later  stage  of 
processing  and  at  a  different  level  of  representation 
CHAN78a. Y0R80D. 

II.  6  Determinati on  of  the  Conditionals 

Our  primary  purpose  in  updating  the  likelihoods  of  edges  is 
to  reduce  uncertainty  in  the  presence  of  elemental  edges  by 
relating  them  to  their  local  contexts.  The  organization  of  edges 
into  boundaries  can  be  viewed  as  the  derivation  of  a  likely  "line 
drawing"  that  is  "close"  to  the  initial  edge  estimates.  and  at 
the  same  time  is  "consistent"  with  desired  properties  of  the  line 
drawings. 

The  conditional  probabilities  in  equation  <3)  provide  the 
theoretical  basis  for  embodying  the  contextual  information  of 
boundary  continuity  in  the  updating  of  edge  probabilities.  The 
conditional  -  probabilities  relate  the  joint  probability 
distribution  of  the  central  edge  with  its  surrounding  context  to 
the  distribution  of  the  surrounding  context;  by  definition 
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P(H1E1 . V  "  P(H‘  El . E6)/P(El,...,E6)  <4) 

These  conditionals  will  be  estimated  Prom  the  class  of  all  line 
drawings  that  are  acceptable  or  desired;  or  in  other  words  the 
class  of  "likely"  line  drawings  that  are  expected  to  be  derived 
by  the  edge  process.  Figure  11  is  an  example  line  drawing 
demonstrating  the  characteristics  of  the  boundaries  that  are 
expected  to  appear  in  our  desired  line  drawings  —  lines  are 
continuous  and  either  terminate  or  meet  at  junctions  with  other 
lines.  One  should  notice  that  a  primitive  edge  element  can 
appear  in  only  a  few  types  of  local  patterns  in  our  class  of  edge 
images.  Isolated  edges  and  lines  with  small  gaps  do  not  appear. 
Any  edge  which  appears  as  part  of  a  boundary  will  have  a  boundary 
continuation  through  at  least  one#  and  probably  both#  vertices  of 
that  edge.  Furthermore#  an  edge  which  participates  in  a  boundary 
continuation  might  take  the  form  of  either  a  single  line 
continuation#  or  the  edge  may  complete  a  line  junction  of  two  or 
more  lines. 


The  conditional  probabi 1 i ties 
sample  of  line  drawings  that  are 
desired  line  drawings.  Mapping 
rectilinear  edge  representation 
large  number  of  local  contexts 


can  be  estimated  from  a  large 
representative  of  the  class  of 
these  drawings  onto  the 
used  in  this  paper  provides  a 
for  estimating  the  joint 
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probabilities  of  equation  (4).  Rather  than  actually  collect  such 
a  set  of  samples  and  exhaustively  tabulate  the  statistics  of  the 
ten  equivalence  classes*  we  can  construct  a  general  model  of  the 
class  of  drawings  and  theoretically  specify  values  of  the 
conditional  probabilities  from  the  model. 

It  is  straightforward  to  tabulate  how  many  times  an  edge  or 
no-edge  occurs  with  each  context  Vij  and  then: 

p<HlV  " P(H*  vij)/p(vij) 

"  P(H*  Vij>/£p(H-T,  Vy)  +  P(H«F,  V  )] 

Figure  12  summarizes  the  joint  occurrence  of  H  with  Vij  for  each 
of  the  10  equivalence  classes  based  upon  our  model  of  desired 
line  drawings. 

The  equivalence  class  of  V00  is  fairly  obvious  in  that  we  do 
not  expect  (or  want)  to  see  "isolated"  edges  —  i.  e.  *  a  single 
edge  with  no  continuation  of  edges  in  its  context.  Thus* 
P<H-T|V00)  is  close  to  zero.  The  Vll  case  is  also  clear  in  that 
the  central  edge  should  be  present  and*  therefore*  P(H»t|vii>  is 
close  to  one. 

The  case  of  V01  is  somewhat  more  subtle.  If  we  consider  a 
boundary  which  terminates  as  a  "line  ending”  (i.e. *  with  no 


Figure  11.  Boundary  Characteristics  Expected  in  Experimental  Domain. 

Example  line  drawing  showing  characteristics  of  boundaries 
and  vertices  expected  in  the  general  class  of  line  drawings. 

(a)  Edges  expected  to  appear.  The  local  edge  contexts  shown 
around  the  V01,  Vll,  V12,  V13  contexts  are  the  only  ways, 
subject  to  symmetry,  that  an  edge  can  participate  in  a  boundary 
in  our  class  of  line  drawings.  (b)  The  edges  marked  V00,  V02, 
and  V03  clearly  do  not  participate  in  closed  boundaries  and 
should  not  appear  in  the  line  drawing.  The  absence  of  edges, 
as  shown  by  open  rectangles,  in  the  Vll,  V12,  and  V13  context 
would  result  in  a  break  in  boundary  continuity  and  these 
cases  also  are  not  expected  to  appear,  (c)  Contexts  with 
inherent  ambiguity.  There  is  some  degree  of  ambiguity  concern¬ 
ing  the  need  for  the  edges  in  the  V22,  V23,  and  V33  contexts. 
None  of  them  are  required  for  boundary  continuity,  although 
they  may  appear  because  they  accurately  reflect  the  underlying 
data  in  the  pixel  perpendicular  to  the  edge.  One  would  also 
expect  these  cases  to  occur  in  dense  edge  areas,  such  as 
heavily  textured  portions  of  an  image,  or,  as  we  have  seen, 
in  gradients  (cf.  Figures  6  and  7). 
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continuity  to  on*  side)*  then  the  last  edge  appearing  in  that 
boundary  will  have  a  VOl  context.  However*  for  each  such  case 
there  are  three  edge  locations  forming  the  VO  vertex  of  that  last 
edge*  which*  when  considered  as  the  central  edge  location 
themselves*  will  contribute  to  the  count  of  H=F  and  VOl*  thus* 
there  are  three  times  as  many  joint  H«*F  and  VOl  occurrences  as 
joint  H*T  and  VOl  occurrences. 

The  cases  of  V12  and  V13  involve  the  edge  locations  leading 
into  the  junction  of  boundaries*  and  they  derive  from  the  fact 
that  a  boundary  is  not  expected  to  terminate  just  before  meeting 
another  boundary.  Finally*  the  V22*  V23.  and  V33  all  involve 
cases  which  are  somewhat  ambiguous.  The  absence  of  the  central 
edge  does  not  really  affect  edge  continuity  in  the  neighborhood 
since  there  are  independent  continutions  to  each  side  of  the 
edge.  However*  if  the  underlying  data  implies  the  presence  of 
the  central  edge*  its  survival  should  not  be  discouraged.  Since 
it  is  much  more  likely  that  two  nearby  boundaries  are  not 
connected  by  "bridging”  edges  (as  in  the  case  of  the  boundaries 
associated  with  narrow  horizontal  or  vertical  stripes)*  a  low 
value  for  P(H»T|  V22)  is  arbitrarily  chosen*  say  .  1.  The  V23  and 
V33  contexts  have  similar  analyses.  Note  that  the  V22*  V23*  and 
V33  contexts  might  be  expected  to  be  more  frequent  in  textured 
areas  or  in  places  of  non-zero  gradients. 


Joint  Occurrence 
of  H  and  Vij 


Join_t  Occurrence 
of  H  and  Vij 


Approximate 
P(H | Vij ) 


voo 

low 

high 

0 

V01 

number  of  line 
endings 

three  times  number  of 
line  endings 

.25 

V02 

low 

high 

0 

V03 

low 

high 

0 

Vll 

high 

low 

1 

VI 2 

high 

low 

1 

V13 

high 

low 

1 

V22 

V23 

V33 

number  of  times  two 
or  more  nearby  lines 
and/or  line  junc¬ 
tions  have  a  bridge 

number  of  times  two  or 
more  nearby  lines  and/ 
or  line  junctions  do 
not  bridge 

«  .5  ? 

(a) 


Vij 

P(H  Vij) 

VOO 

0 

V01 

.25 

V02 

0 

Vll 

1 

V12 

1 

V22 

.1 

(b) 

Figure  12.  Estimation  of  the  Conditional  Probabilities  from  our  Model  of 
the  Class  of  Line  Drawings.  (a)  The  ten  equivalence  classes, 
(b)  Reduction  to  six  equivalence  classes  based  upon  identical 
semantics  of  the  V2  and  V3  vertices. 
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Th *  set  of  ten  equivalence  classes  of  local  edge  contexts 
can  be  further  reduced.  An  examination  of  Figure  12<a)  shows 
that  the  conditionals  associated  with  the  V2  and  V3  contexts  are 
the  same.  This  reduces  the  set  of  equivalence  classes  to  six* 
namely  Vij*  0  <  i  £,  j  <  2*  as  shown  in  Figure  12(b)*  where  the  V2 
vertex  class  now  includes  V3  also. 


II.  7 


Hut  giiimta  fi£  Boundaru 


It  may  be  helpful  to  briefly  examine  the  semantics  of  these 
six  equivalence  classes  over  the  64  patterns  of  edge  contexts: 


P(H»Tj VOO)  »  0 
P(H»T|  V01)  -  .25 

P ( H=T i V02 )  ■  0 

P(H-Tlvil)  -  1 
P(H»T|V12)  ■  1 


an  "isolated  edge"  which  does  not  appear 
to  participate  in  a  continuous  boundary* 

an  "ambiguous  boundary  continuation" 
where  the  line  may  terminate  or  else 
continue  in  one  or  more  of  the  three 
possible  directions; 

a  "spur"  which  is  an  edge  whose  absence 
will  not  affect  the  boundary  continuity 
of  the  other  edges  which  are  present  in 
the  context; 

local  "continuity"  for  a  single  boundary 
is  ensured  through  the  central  edge; 

boundary  continuation  via  "junction 
completion"  is  ensured  through  the 
central  edge; 

"ambiguity  of  bridge"  linking  nearby 
boundaries;  the  edge  context  provides 
no  information  concerning  the  presence 
of  the  central  edge. 


I 


i 


P(H-T|  V22)  -  .  1 


Hopefully*  the*#  descriptions  provide  some  insight  into  the 
semantics  of  local  patterns  of  continuity.  Four  of  the  patterns 
—  VOO»  V02*  Vll*  V12  —  appear  to  be  quite  reasonable.  However* 
uncertainty  in  the  VOl  and  V22  cases  will  give  us  some  degree  of 
difficulty.  In  the  VOl  case  the  problem  is  immediately  obvious 
when  one  considers  that  the  abrupt  termination  of  a  boundary 
composed  of  elemental  edges  with  probability  one  will  update  the 
probability  of  all  three  adjacent  edges  to  a  value  of  .25.  This 
appears  to  be  a  diffusion  of  information  rather  than  a  coherent 
organizing  process. 

There  is  also  a  potential  problem  in  the  use  of  the  V22 
conditionals  in  the  case  of  two  lines  which  are  close  together. 
For  example*  consider  two  strong  parallel  lines  one  pixel  apart* 
as  in  the  case  of  a  black  stripe  on  a  white  background*  the 
probability  of  an  edge  linking  the  right  and  left  boundary  of  the 
stripe  at  each  possible  "bridging"  location  will  be  set  to  a 
constant  (say  K>,  no  matter  what  is  implied  by  the  underlying 
data  of  the  pixels  in  the  stripe.  Thus*  any  value  of  K*  high  or 
low*  will  be  wrong  in  certain  cases!  These  issues  will  be  set 

j  * 

aside  for  the  moment  but  will  be  addressed  again  later. 


ii.  8  fttiimUgn  Uaiatinfl  n  consistent 


The  development,  thus  far.  has  focused  on  determining  the 
likelihood  of  a  single  central  edge  given  its  local  context  of 
neighboring  edges,  and  was  based  to  a  reasonable  degree  on  a 
Bayesian  view  of  this  relationship.  Now  we  wish  to  embed  this 
relationship  within  an  iterative  updating  scheme  which  is  to  be 
applied  simultaneously  to  all  edge  locations  in  the  image, 
realizing  full  well  that  this  step  takes  us  outside  the 
contraints  imposed  by  Bayesian  theory.  This  parallel  updating 
will  be  viewed  as  a  relaxation  labelling  process  whereby  an 
inconsistent  set  of  likelihoods  are  driven  towards  consistency 
with  their  neighborhoods.  The  conditional  probabilities  serve  as 
compatibility  coefficients  in  relating  the  neighborhood 
influences  of  edges  in  the  context. 

Since  the  parallel  updating  scheme  is  to  be  repeatedly 
applied.  it  will  be  convenient  to  extend  the  notation  to  reflect 
the  iterative  progression  of  the  computation  by  including  the 
notion  of  time  in  the  updating  function: 

Pt+1(H)  -  H  PC(H|V  )P*(V .,)  (5) 

i  j  1J 

Let  us  make  the  following  important  simplifying  assumption:  the 
conditional  probability  distribution  of  H  upon  its  context  is 


given  and  is  stationary.  Consequently  the  conditionals  in 

equation  <5)  will  not  require  a  superscript  t.  Me  now  observe 

'  v 

that  the  unary  marginal  probabilities  of  edges  P(Ei)  are 
determined  a  priori  on  the  basis  of  external  physical  evidence> 
but  the  joint  probability  of  edges  P<E1»  E2>  E3»  E4»  E5*  Eh)  are 
not  available.  This  implies  that  the  joint  probability  cannot  be 
computed  without  additional  information  or  assumptions*  the  most 
obvious  being  independence  of  the  edges  in  the  context.  By  using 
the  assumption  of  independence  of  left  and  right  vertices  to 
approximate  the  joint  probability  of  edges*  yielding 

Pt+1(H)  -  l  l  P(H|Vij)Pt(Vi)Pt(Vj)  <&> 

We  can  further  approximate  the  probability  of  vertex  types  using 
independence  of  edges  in  each  vertex 

P(V0)  -  PCEj-F,  E2«F,  E3-F)  -  PCEj-F)  P(E2-F)  P(E3-F) 

P(VX)  -  P(E1-T,  E2«F,  E3-F) 

+  PCEj-F,  E2-T,  E3-F) 

+  P(EX-F,  E2-F,  E3-T) 

P(V2)  -  P(E1-T)  P(E2-T)  P(E3-F) 

+  PCEj^-T)  P(E2-F)  P(E3«T) 

+  PCE^F)  P(E2-T)  P(E3-T) 

P(V3)  -  P(Ej-T)  P(E2-T)  P(E3-T) 


The  assumption  of  edge  independence  may  cause  the  careful  reader 
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4 


some  concern.  We  are  using  the  dependencies  between  every  edge 


and  its  neighboring 

context  to 

update  edge 

probab i 1 i ties* 

yet 

portions  of  the 

context  are 

assumed  to 

be  independent! 

In 

particular  when  one 

of  the  edges 

*  say  El*  is 

considered  as 

the 

central  edge*  then  the  updating  function  will  be  based  on  the 
dependencies  of  El  upon  E2  and  E3  and  itf;  other  surrounding  edges 
in  the  context  —  yet  we  have  already  assumed  El*  E2*  and  E3  are 
independent.  Thus*  we  have  violated  theory  here  and  have  entered 
the  realm  of  heuristics.  Even  without  the  assumption  of 
independence*  our  mechanism  for  parallel  updating  is  still  not 
theoretically  sound  from  a  Bayesian  viewpoint.  While  any  single 
context  could  be  used  to  update  an  edge  probability*  that  central 
edge  should  not  also  be  used  to  update  the  value  of  the  other 
edges  which  were  in  its  context.  Otherwise*  this  allows  the 
probability  of  an  edge  in  equation  <5>  to  have  an  effect  upon 
itself  as  closed  loops  of  probability  updates.  An  examination  of 
the  theoretical  issues  in  this  approach  to  relaxation  labelling 
and  the  propagation  of  information  in  inference  networks  CDUD761 
is  treated  in  detail  in  CL0W803. 

Despite  some  of  the  theoretical  difficulties*  we  will 
examine  the  use  of  such  an  updating  function.  In  general* 
initial  values  of  individual  edge  prnbabi 1 i ties  are  consistent 
with  the  underlying  feature  data  which  gave  rise  to  them*  but  are 


not  consistent  with  their  surrounding  contexts  in  terms  of  the 
conditional  probabilities  derived  from  the  class  of  ideal  line 
drawings.  If  the  conditionals  remain  constant*  then  one  can  view 
the  updating  process  as  a  means  by  which  the  likelihood  of  a 
central  edge  becomes  consistent  with  its  surround. 

If  the  array  of  edge  probabilities  ever  reaches  a  state 
where  the  updating  function  leaves  every  edge  probability  at  its 
current  value*  i. e. *  Pt+1<H)  *  Pt(H)  for  all  H  in  the  image*  then 
the  system  is  at  a  fixed  point.  Further  iterative  updating  will 
leave  all  values  unchanged.  Note  also  that  at  a  fixed  point  all 
edges  are  consistent  with  their  local  surrounding  contexts 
because  the  updated  values  implied  by  the  contexts  are  the  same 
as  their  current  values.  Thus*  fixed  points  are  consistent  in 
this  pseudo-Bayesian  sense. 


II. 9  Examination  of  Specific  Cases 


In 

this 

section 

we  examine  two 

cases 

of 

initial 

probabilities 

which 

demonstrate  certain 

difficulties 

in  our 

formulation  of 

edge  relaxation*  and  then 

present 

some 

results 

which 

confirm 

these 

difficulties.  One 

case  demonstrates  the 

decay  of  edge  patterns  which  should  remain  stable*  while  the 
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other  case  demonstrates  the  growth  of  'noise'  which  should 
disappear. 

Case  A  closed  loop  of  edges  with  probability  q  on  a 

background  of  zero  probability  edges. 

Note  that  q  ■  0  and  q  -  1  are  fixed  points.  However.  if 
O  <  q  <  1,  a  fixed  point  is  approached  in  the  updating  process/ 
but  there  is  total  loss  of  the  starting  information  that 
represented  the  boundary  (refer  to  Figure  14).  Let  us  analyze 
why  this  takes  place  by  representing  P(H=T|V01)  with  variable  W. 
Then  the  updating  equation  for  central  edges  in  the  closed  loop 
(each  with  probability  q)  is  given  in  Figure  13(a).  while  the 
update  for  zero  probability  edges  that  ore  touching  the  boundary 
are  given  in  Figure  13(b). 


It  is  interesting  to  note  here  that  a  value  of  W  -  .  25.  as 
defined  by  our  estimates  in  Figure  12.  will  cause  the  updated 
value  of  edges  H  in  the  loop  to  decrease  in  probability  if  q  <  l. 
even  if  q  is  arbitrarily  close  to  li  thus 

Pt"*"1(H*»T)  <  PC(H-T)  when  PC(H«T)  <  1. 


At  the  same  time,  edges  adjacent  to  the  boundary  of  probability  q 
will  change  to  a  non-zero  edge  probability.  As  the  updating 
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process  continues*  the  information  is  diffused*  as  shown  in 
Figure  14a>  until  it  approaches  a  fixed  point.  All  values  of  q 
other  than  0  and  1  will  not  allow  the  initial  pattern  to  remain 
stable. 

Simple  examination  of  the  updating  equation  shows  that  a 
value  of  M  ■  .5  will  allow  the  edges  in  the  loop  to  remain 
constant.  A  value  of  W  >  .  S  will  allow  edges  in  the  loop  to 
increase*  that  is.  P  t+1(H=T)  >  Pt(Hs»T>.  In  either  case*  however* 
the  zero  probability  edges  hanging  off  the  boundary  will  still 
increase  and  eventually  the  information  is  diffused  as  shown  in 
Figure  14<b)  and  <c). 

Case  g:  A  uniform  field  of  low  equiprobabi 1 ity  edges  q. 

In  this  case  all  edge  locations  have  an  identical  context  in 
the  updating  process.  If  q  has  a  low  value*  one  might  consider 
it  desirable  for  all  edge  probabilities  to  converge  to  a  fixed 
point  of  0.  Unfortunately.  our  formulation*  which  has  been 
guided  by  theoretical  considerations*  is  not  so  well-behaved. 
Given  the  analysis  of  Case  1*  let  us  set  the  value  of  P(H»T|V01> 
to  .5  and  determine  the  updated  value  at  all  edges  as  a  function 
of  q. 
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lJ(b) 


glare  13.  Case  1  Analysis,  (a)  Update  equation  for  a  central  edge 
in  a  closed  loop  of  edges  with  probability  q.  The  vertex 
probabilities  are  used  to  compute  the  probability  of  V00,...,V22 
edge  contexts,  from  which  the  update  equation  can  be  obtained, 
(b)  Analysis  similar  to  (a)  but  for  the  edges  adjacent  to 
the  closed  loop. 
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Figure  14.  Case  1:  Closed  Loop  of  Edges.  Each  edge  in  the  loop 
has  an  initial  probability  of  .8  while  background 
edges  are  of  probability  0.  t-.ach  image  shows  the 
original  data  (upper  left  corner)  followed  by  the 
edge  relaxation  after  iterations  2  (upper  right),  5 
(lower  left),  and  10  (lower  right).  The  parameters 
used  other  than  P(H/V01)  were  those  given  in  Figure 
12(b). 
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First*  it  is  helpful  to  examine  the  probabilities  of  the 
four  vertex  types  VO*  VI*  V2*  and  V3*  given  a  set  of  three  edges 
each  of  probability  4.  Figure  15(a)  lists  the  vertex  likelihoods 
for  several  low  values  of  q.  With  a  value  of  q  *  .  2*  the 
likelihood  of  VI  is  relatively  high:  .384.  Interestingly*  this 
causes  the  updating  of  the  central  edge  to  be  more  heavily 
weighted  by  V01  than  V00  because  P(VOl)  >  P(VOO).  Figure  15(b) 
depicts  the  relative  contribution  of  each  context  Vij  in  updating 
the  central  edge  to  .424.  Thus*  in  a  single  iteration  a  uniform 
field  of  edges  with  probability  .2  would  be  raised  to  .424  and 
then  even  higher  on  the  next  iteration.  This  process  converges 
to  a  fixed  point  about  .65. 

While  this  increase  of  low  probability  edges  may  disturb  the 
intuition  of  some*  the  reason  is  that  P(VOO)  *  (l-q)**6  and  for  q 
■  .  1*  .2*  and  .3*  P(VOO)  equals  .53*  .26*  and  .12  respectively. 
Thus*  a  context  of  six  low  probability  edges  can  lead  to  a 
relatively  high  likelihood  of  the  VOl  case*  because  it  can  become 
quite  unlikely  that  all  six  edges  are  absent.  The  updating 
process  increases  the  probability  of  the  central  edge  in  response 
to  the  many  uncertain*  but  possible*  ways  of  extending  the 
boundary. 


In  order  to  examine  th#  impact  of  those  problems  on  on 
actual  scene#  the  relaxation  algorithm  tees  applied  to  the  bosh 
subimage  (Figure  lb).  This  sub  inage  nos  chosen  because  it 
exhibits  a  range  of  characteristics  including  clear  straight 
lines  at  several  orientations*  relatively  weak  edges  in  many 
places*  texture  edges*  house  edges  in  the  textured  areas# 
non-zero  intensity  gradients  of  several  foras*  etc. 

The  results  of  the  edge  update  process  after  iterations  0* 
1#  2#  and  5*  using  the  conditionals  described  in  Figure  12b*  are 
shown  in  Figure  16.  The  results  are  obviously  poor*  and 
improvements  were  not  achieved  by  variation  in  the  conditionals. 
Clear  edge  patterns  immediately  decay  with  diffusion  into  the 
surround*  while  there  is  little  control  in  the  growth  of  low 


values  of  edge  probability  —  which  for  convenience  we  shall  call 
"noise"  here  even  though  the  source  of  this  data  is  varied. 
Attempted  remedies  for  controlling  the  low  probability  edges  via 


techniques  involving  fuziy  probabilities  and  thresholding  also 
were  not  successful. 

It  is  difficult  to  determine  the  source  of  the  problems  in 
the  formulation.  Clearly#  the  most  obvious  theoretical  problems 
are  the  assumption  of  independence  which  allows  the  joint 
probability  of  the  context  to  be  decomposed*  and  the  simultaneous 


Results  from  Edge  Update  Process.  (a)  Initial 
probabilities.  (b)  After  1  iteration.  (c)  2 
iterations.  (d)  5  iterations.  By  the  second 
iteration,  the  diffusion  of  the  original  data  is  well 
underway.  The  uncontrolled  growth  continues  in 
subsequent  iterations  until  the  original  edge 
information  is  gone. 
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and  iterative  updating  of  all  edges.  Rather  than  question  these 
assumptions  upon  which  our  basic  approach  rests*  in  the  next  two 
sections  we  incorporate  additional  forms  of  contextual 
information  and  explore  variations  in  the  updating  equation  as  we 
seek  to  develop  a  formulation  which  performs  as  expected. 
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III.  PARALLEL  EDGE  SUPPRESSION  AND  CONTRAST  DIRECTIONALITY 

Up  to  this  point,  the  definition  of  the  local  context  to  be 
used  in  updating  was  based  primarily  on  the  requirements  imposed 
by  the  continuity  assumption.  The  local  context  defined  thus  far 
is  the  smallest  meaningful  neighborhood  for  an  edge  which  permits 
the  translation  of  the  continuity  assumption  into  a  set  of 
constraints  defined  over  the  neighborhood.  However,  the  presence 
of  edges  defined  by  non-zero  feature  gradients  means  that  an 
extension  of  the  local  context  to  include  the  two  adjacent  edges 
parallel  to  the  central  edge  <edges  E7  and  E8  in  Figure  17a)  will 
provide  highly  relevant  information.  These  locations  represent 
alternative  placements  for  the  collected  gradient  edges  (Section 
II.  3).  Their  absence  makes  it  impossible  to  control  the 
selection  of  that  edge  location  (within  the  set  of  alternative 
choices  of  the  gradient)  which  best  supports  continuity  in  the 
local  context. 

Without  the  expanded  local  context,  in  the  current  system 
all  parallel  edges  could.  and  often  do.  find  support  in  their 

,i  * 

context  and  grow  to  their  maximum  strength.  The  overall  effect 
varies.  depending  .  on  the  situation.  In  the  worst  case,  most 
non-zero  edges  in  the  image  grow,  resulting  in  a  dense  mesh  of 
edges  over  much  of  the  image,  as  shown  in  Figure  16.  Some  form 
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Figure  17.  Extension  of  the  Local  Neighborhood,  (a)  The  local  neighborhood 
is  extended  to  include  the  two  edges  E7  and  E8  parallel  to  the 
central  edge.  This  extension  permits  gradient  information  to 
propagate  into  the  context,  (b)  Ey  maintaining  contrast 
directionality  of  the  edges  relative  to  the  central  edge,  the 
effects  of  parallel  edges  on  the  central  edge  are  easily 
decomposed  into  two  classes  G-ON  and  G-OFF.  For  case  G-ON, 
the  effect  is  inhibitory  and  suppression  is  on;  for  case  G-OFF, 
the  edges  are  parts  of  different  boundary  gradients  and  no 
suppression  takes  place. 
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of  parol  lei  edge  suppression  (or  gradient  suppression)  via 
lateral  inhibition  is  necessary  to  remedy  this.  Adjacent 
parallel  edges  can  be  structured  to  compete  for  survival»  with 
the  outcome  decided  by  the  support  from  their  respective  local 
contexts. 

The  process  of  parallel  edge  suppression  must  take  into 
account  the  direction  of  contrast  of  parallel  adjacent  edges. 
Otherwise  a  one-pixel  wide  black  stripe  on  a  white  background* 
for  example*  would  cause  the  two  distinct  boundaries  of  the 
stripe  to  compete  until  only  a  single  boundary  remained*  a  result 
that  is  clearly  undesirable.  Consequently*  parallel  (and 
adjacent)  edges  should  compete  to  become  the  representative  of  a 
given  gradient  edge*  but  this  process  should  be  deactivated  if 
the  parallel  edges  represent  different  boundaries. 

The  contrast  directionality  of  edges  in  the  initial  data 
provides  sufficient  information  for  the  decision  of  activation  of 
parallel  edge  suppression  for  every  pair  of  parallel  edges.  In 
fact  it  is  precisely  this  feature  of  edges  that  is  used  in  the 
gradient  collection  algorithm  to  group  contiguous  parallel  edges 
(or  equivalent  runs  of  consecutive  pixels)  into  units  to  be 
considered  as  potential  gradient  edges.  Thus*  the  direction  of 
feature  contrast  change  will  be  maintained  as  a  sign*  or 


The  required  properties  of  the  edge  suppression  process  are: 

1.  the  central  edge  will  be  inhibited  when  edge  E7  or  E8 
possesses  the  same  sign  of  contrast  change  as  the 
central  edge; 

2.  parallel  suppression  will  be  decoupled  and  there  will  be 
no  effect  from  edges  E7  or  E8  (or  both)  if  they  have  a 
different  contrast  sign#  and 

3.  the  degree  of  parallel  suppression  will  be  directly 
proportional  to  the  likelihood  of  the  parallel  competing 
edge(s). 

For  simplicity  the  effect  of  parallel  edge  suppression  will 
be  separated  from  the  contextual  influence  of  boundary  continuity 
by  introducing  a  heuristic  assumption  of  independence  between  the 
parallel  edges  E7>  E8»  and  the  six  continuing  edges  in  Vij. 

* 

|  Thus# 

f 

L  P<a-T)  - HI 

*  1  1  h 

may  be  rewritten  as: 

P(H-T)  -HI 

i  j  e7 


l  P  (H*T  j  Vjj  ,  E?,  Eg)  P(V±jt  E?,  Eg) 


l  P(H-T|Vir  E?,  Eg)  PCV^)  P(E?)  P(Eg) 
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The  key  problem  remaining  is  the*  determination  of  the 
conditionals  relating  the  central  edge  to  the  eight  edges  in  the 
context.  One  approach  is  to  enumerate  the  256  cases  of  the  joint 
presence  and  absence  of  these  edges  and  estimate  the  associated 
conditionals  from  the  model  of  desired  line  drawings.  The 
analysis  is  further  complicated  by  the  need  to  include  the 
contrast  directionality  for  controlling  suppression  of  parallel 
edges*  thereby  significantly  increasing  the  number  of  cases  to  be 
considered.  The  approach  employed  here  is  simpler. 

First*  let  us  note  that  the  edges  E7  and  EB  can  each  be  in 
one  of  three  possible  states:  1)  no  edge*  labelled  "F"*  2)  an 
edge  of  the  same  contrast  direction  as  the  central  edge*  labelled 
"S".  or  3)  an  edge  of  the  opposite  contrast  as  the  central  edge* 
labelled  "O".  There  are  nine  possible  joint  occurrences  of  the 
states  of  E7  and  E8.  These  are  divided  into  two  groups  G-ON  and 
G-OFF.  corresponding  to  the  cases  of  parallel  edge  suppression 
being  activated  or  deactivated*  respectively*  as  shown  in  Figure 
17<b).  The  contrast  signs  of  edges  are  fixed  during  initial 
gradient  collection*  and  consequently  only  some  of  these  nine 
cases  are  relevant*  in  particular  only  one  of  the  pair  of  signs 
<+,  +)»  (+*  -),  <-*  +)»  and  <-.  -)  for  any  pair  of  edges  can  have 
a  non-zero  probability*  while  P(E7»F)  for  example  can  be  non-zero 
no  matter  what  the  sign  of  edge  E7.  Th<*  probability  of  the  nine 


cases  will  vary  depending  upon  the  signs  and  magnitudes  of  P<E7) 
and  P<EB)>  and  can  be  computed  as  a  product  of  the  presence  or 
absence  of  the  two  edges. 

Inclusion  of  the  parallel  edges  E7  and  EB  in  the  local 
context  of  the  central  edge  can  be  viewed  in  terms  of  G-QN  and 
G-OFF  instead  of  E7  and  EB: 

P(H-T)  =  ll  [P(H-T|V  ,  G-OFF)  *  P(V..,  G-OFF) 
i  j  ^ 

+  P(H-T|V±j,  G-ON)  *  P(vi;j.  G-ON)] 

and  by  the  independence  of  Vij*  and  E7  and  EB 

P(H-T)  -  l  l  [P(H«T|V  ,  G-OFF)  *  P(V  )  *  P(G-CFF) 
i  j  J 

+  PCH-Tlv^,  G-ON)  *  p(vij)  *  P(G-ON)  ] 

The  required  effect  of  cases  G-ON  and  G-OFF  suggest  a  rather 
simple  means  of  integrating  their  effect  into  the  conditionals. 
Case  G-OFF  should  have  no  effect  on  the  updating;  therefore 

P(H|V±j,  G-OFF)  *  P(H|Vi;J) 

For  case  G-ON  an  assumption  of  conditional  independence  of  Vij 
and  G-ON  given  H«T  leads  to  a  simple  embodiment  of  parallel 
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P(V  ,  G-ON|h»T)  *  P(H-T) 
P(H-TlV  '5-0«  '  3p(Vir  G-ON) - 

PCV^Jh-T)  *  P(G-ON|H-T)  *  P(H-T) 
P (Vi;J)  *  P(G-ON) 

P(H-T|Vi1)  *  P(H«T|G-ON) 

P(H-T) 


-  p(h-t|v1j) 


*  P(H-T[G-ON) 
P(H-T) 


If  we  let 


c  _  P(H»T| G-ON) 
P(H-T) 


where  0  £  C  £  1 


then*  a  setting  of  constant  C  to  a  value  less  than  one  has  the 
desired  effect  of  parallel  edge  suppression  using 

P(H-T|Vijt  G-ON)  -  C  *  P(H-T|V1:)) 

for  all  vertex  classes  Vij.  This  parameter  will  allow  variable 
control  in  the  overall  effect  of  the  suppression  mechanism. 
Thus*  we  get 

P(H-T)  "ll  [P(H-T|V  )  C  P(V  )  P(G-ON) 
i  j  J  J 

+  P(H-T|Vi;J)  P(Vi;J)  P(G-OFF)  ] 

finally  yielding 

P(H-T)  -  [C  *  P(G-ON)  +  P(G-OFF) ] 

*  [l  l  P(H-T|V  )  P(V  ) ] 
i  j  J 


Note  that  the  effect  of  parallel  edge  suppression  and 
boundary  continuity  have  been  decomposed  into  product  terms.  The 
second  term  is  exactly  the  update  before  parallel  suppression  was 
considered!  while  the  effects  of  parallel  suppression  have  been 
isolated  in  the  first  term.  The  required  effects  of  gradient 
suppression  can  now  be  attained.  As  condition  C-GFF  becomes  more 
certain/  the  value  of  the  first  term  approaches  1.0  which  is  the 
case  without  parallel  suppression.  Alternatively  as  condition 
G-ON  becomes  dominant  because  of  likely  parallel  competing  edges# 
the  value  of  the  first  term  goes  to  C.  If  C  *  1#  parallel 
suppression  is  turned  offi  for  C  «  O#  maximal  suppression  is 
achieved.  In  practice  a  value  of  C  =  .  5  appears  reasonable  given 
the  particular  gradient  collection  algorithm  in  use. 


IV.  REFORMULATION  OF  THE  LOCAL  CONTEXT  TO  INCLUDE  THE  CENTRAL  EDGE 


In  Section  II  of  this  paper  the  relaxation  process  for 
updating  the  probability  of  edges  teas  motivated  and  guided  by 
considering  the  Bayesian  relationships  of  edges  in  a  local 
context.  Empirical  results>  however,  were  quite  disappointing. 
One  apparent  difficulty  to  be  explored  in  this  section  is  the 
lack  of  direct  influence  of  the  current  central  edge  probability 
on  the  updated  central  edge  probability.  While  it  is  clear  that 
the  current  probability  of  a  central  edge  will  have  an  indirect 
effect  upon  itself  over  successive  iterations  by  influencing  the 
edges  in  its  context  which,  in  turn,  influence  it.  the  updating 
process  fails  to  capture  meaningful  information  in  a  single 
updating  of  a  local  context.  For  example  in  the  case  of  a 
boundary  termination,  the  three  edges  which  are  absent.  as  well 
as  the  last  edge  actually  present  in  the  boundary,  will  be 
updated  to  similar  values  because  all  four  of  the  edges  see  a  VOl 
context.  There  is  a  lack  of  discrimination  between  the  last  edge 
in  the  boundary  and  the  adjacent  absent  edges.  Much  of  the 
relevant  information  that  is  being  ignored  is  available  in  the 
current  probability  of  the  central  edge. 

We  will  present  two  formulations  th<»t  include  the  central 
edge  directly  in  the  updating  process.  They  are  both  heuristic 
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as  art  most  of  the  approaches  in  this  paper  when  viewed  from 
strict  theoretical  grounds.  The  first  is  of  some  interest 
because  it  is  theoretically  motivated  by  a  loose 
re-interpretation  of  the  Bayesian  formulation*  but  it  still  does 
not  produce  effective  results.  The  second  formulation  involves 
updating  of  both  edge  and  no-edge  labels  with  normalization.  It 
retains  the  same  values  of  conditional  probabilities  that  were 
originally  suggested  theoretically*  while  capturing  the  effect  of 
competition  between  labels  via  normalization.  Highly  effective 
results  are  achieved  in  the  second  formulation. 


Formulation  A;  Edges  as  Distinct  Events  Over  Time 

It  is  difficult  to  justify  a  Bayesian  formulation  which 
includes  the  central  edge  in  the  context  unless  we  modify  our 
basic  definition  of  the  events  whose  probability  distributions 
are  being  manipulated.  Up  to  this  point  we  have  associated  a 
probability  with  each  edge  location*  treating  each  edge  as  a 
particular  event  on  the  spatial  array  of  image  points.  The 
problem  then  becomes  one  of  updating  the  probability  of  these 
events  so  that  consistency  with  the  set  of  fixed  conditional 
probabilities  is  achieved.  Thus*  one  can  view  the  updating 


process  as  adjustment  (over  time  or*  equivalently*  iteration 
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number)  of  the  original  estimates  of  the  probability  of  edge 
presenc  e. 

A  different  interpretation  of  these  events  may  be  obtained 
by  viewing  as  distinct  events  edges  which  are  in  the  same  spatial 
location#  but  are  updated  over  time.  Thus/  edges  at  time  t  and 
t+1  in  the  same  spatial  location  will  be  considered  distinct 
entities.  However#  local  neighborhoods  and  conditional 
probabilities  will  still  be  used  to  relate  events  in  successive 
iterations.  The  updating  process  becomes  one  of  deriving  a 
probability  distribution  of  another  set  of  edges  which  has  been 
derived  from  the  probability  distribution  Df  the  original  edge 
array.  In  fact#  it  is  natural  to  think  of  both  the  initial  and 
final  probability  distribution  of  edges  to  be  available  to 
further  processing#  the  initial  array  represents  sensory  data 
while  the  final  array  represents  a  boundary  continuity 
interpretation  of  that  data. 

The  superscript  t  is  now  related  to  the  random  variable  of 
the  edges  themselves  instead  of  the  probability  of  the  random 
variable.  The  relationship  between  the  probability  of  the 
central  edge  at  time  t+i  and  the  seven  edges  in  our  local  context 


at  time  t  is  then: 
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P(Ht+1-T)  -ll  [P(Ht+1-T,  H^T,  VC  ) 
i  j 

+  P(Ht+1-T,  HC«F,  )J 

P(Ht+1-T)  +  ll  [PtH^-TlH^T,  VC  )  P(HC-T,  VC.,) 
i  j 

+  pch^-tIh^f,  vtij)  PO^-F,  v^)] 

Note  that  the  same  theoretical  difficulty  in  estimating  the 
joint  probabilities  of  edges  still  persists.  Only  the  unary 
marginals  are  available*  yet  we  know  that  the  edges  in  the 
context  are  not  independent.  Nonetheless*  for  lack  of  a  better 
solution*  an  assumption  of  independence  gives  us 

P(Ht+1-T)  -  l  l  [P(Ht+1-T|HC-T,  V*^)  P(VCi;j)  P(hVt) 

1  P(Ht+1-T|HC-F,  VC  )  P(VCi:j)  P(HC-F)] 

P(Ht+1-T)  -  l  l  P(VC  )  [P(Ht+1-T|Vt  ,  H^T)  *  P(HC-T) 
i  j 

+  P(Ht+1-T|vt  ,  Ht“F)  *  P(HC-F)] 


The  key  difference  in 

this 

updating  formula 

is  that 

the 

conditionals 

are  based 

upon 

seven  edges  in  the 

context* 

this 

introduces  a 

multiplicative  factor  involving  the 

probability 

of 

the  central  edge  on  the  previous  iteration. 

Figure  18  summarizes  the  new  set  of  conditionals  based  on 
estimates  from  the  desired  line  drawings  discussed  earlier.  The 
six  equivalence  classes  Vi j  have  now  become  12  because  the 
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0 

Figure  18. 


Estimation  of  Conditional  Probabilities  with  Central  Edge 
in  the  Context  (Formulation  A).  The  conditionals  are'derived 
from  the  model  of  desired  line  drawings  described  in  Figure  11 


t 
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central  edge  mag  be  present  or  absent  with  each.  It  is  quite 
interesting  to  note  that  all  conditionals  are  0  or  1  in  the  seven 
edge  context.  It  appears  that  inclusion  of  the  central  edge  has 
greatly  clarified  the  desired  states  in  the  updating  process. 

In  the  case  of  the  VOO#  V02#  Vll#  and  V12  contexts#  the 
conditionals  are  the  same  whether  the  central  edge  is  present  or 
absent.  Thus#  in  these  cases  there  is  effectively  no  change  in 
the  updating  because  the  central  edge  will  factor  out  of  the  new 
updating  formula.  However#  in  the  V22  cose  the  values  of  1  and  0 
for  P (Ht+l^TI Ht»T*  Vij)  and  P<Ht+lj  Ht«F»  Vij)»  respectively# 
cause  a  linear  interpolation  between  the  values  for  P<Ht*T>  and 
P<Ht»F).  When  P<Vt22)  -  1  then 

P(Ht+1=T)  =■  P(Ht"T) 

which  appears  to  be  exactly  what  we  want.  The  central  edge  will 
stabilize  at  the  current  probability  that  was  the  best  estimate 
based  on  local  evidence#  if  there  was  little  evidence  for  a 
bridging  link  between  lines#  the  edge  probability  can  remain  low. 

In  the  previous  formulation#  the  VOi  context  also  caused 
some  difficulty  in  defining  the  appropriate  conditionals.  This 
was  due  to  the  symmetry  in  the  cases  where  the  central  edge  was 
present  and  where  it  was  absent;  it  was  impossible  to 
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distinguish  these  casts*  gat  it  was  clearly  desirable  to  do  so. 
Inclusion  of  tha  central  edge  in  the  conditional  allows  the  last 
edge  present  in  the  boundary  to  be  distinguished  from  the  three 
adjacent  edges  that  are  absent.  Thus*  a  boundary  termination  in 
terms  of  the  four  edge  probabilities  is  stable  in  the  updating 
process  when  the  boundary  has  probability  one.  There  will  be  no 
diffusion  of  the  information  in  the  case  of  a  boundary  with  edges 
of  probability  one  that  abruptly  terminates. 

Once  again  experimental  results  demonstrate  that  intuition 
can  be  quite  deceiving*  particularly  in  the  case  of  parallel 
local  updating  of  complex  information.  Figure  19  depicts  several 
iterations  of  the  seven-edge  context  using  the  conditionals  of 
Figure  18  and  no  gradient  suppression.  The  end  result  of  this 
process  is  the  diffusion  and  growth  of  edges  across  much  of  the 
image.  Figure  20  depicts  runs  with  moderate  and  strong  parallel 
edge  suppression*  and  while  this  algorithm  appears  marginally 
better  than  the  basic  approach  of  Section  II  and  Figure  16*  again 
no  effective  results  are  achieved.  The  dependency  of  results  on 
particular  values  of  conditionals  was  explored  by  varying  several 
values  of  conditionals*  but  this  parameter  diddling  yielded  no 
significant  improvement. 
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Fiaur»  19.  Edge  Results  far  Formulation  A:  Inclusion  of  Central 

Edge.  For  these  result;-,  the  conditionals  shown  in 
Figure  IS  were  used  and  no  grodient  suppression  was 
performed.  The  results  are  discouraging, 

particularly  when  compared  to  ,’igure  16. 


Figure  20.  Edge  Results  for  Formulation  A:  Inclusion  of  Central 

Edge.  Results  for  <a>  moderate  and  <b)  strong 
parallel  edge  suppression  after  10  iterations  exhibit 
a  similarity  to  those  in  Figure  19.  Evidently  the 
poor  results  are  not  sensitive  to  the  degree  of 
parallel  edge  suppression.  Similar  explorations  of 
the  dependency  of  the  results  on  other  parameter 
settings  yielded  no  significant  change  in  their 
qual i ty . 
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Part  of  the  problem  of  rapid  growth  of  edges  in  Figure  19 
suprisingly  comes  from  the  V12  case.  Figure  21  is  an  analysis  of 
the  effect  of  updating  given  that  one  uncertain  boundary  at  a 
probability  of  .5  is  in  the  vicinity  of  a  boundary  whose 
probability  is  one.  Analysis  shows  that  the  central  edge  will  be 
updated  to  at  least  .5  no  matter  what  the  probability  of  the 
central  edge  is*  and  that  the  problem  is  due  to  the  context  of 
Ht=F  and  V12.  The  probability  of  the  central  edge  is  a  factor  in 
one  term*  but  the  probability  of  the  absence  of  the  central  edge 
is  the  multiplicative  factor  in  the  other  term*  and  therein  lies 
the  difficulty.  Again  we  have  a  problem  whereby  the  probability 
of  the  central  edge  is  changing  rapidly  with  little  relationship 
to  the  value  on  the  previous  iteration. 


Formulation  B:  Updating  Edoe  and  NoHEdae  Labels  with 

Nqrmjluati-ffn, 


The  failure  of  Formulation  A  to  produce  acceptable  results 
is  discouraging.  The  probability  of  the  central  edge  appears  to 
change*  almost  independently  of  the  actual  data.  Le.t  us  consider 
the  second  updating  formulation  obtained  by  treating  the  central 
edge  and  its  context  as  two  separate*  independent  sources  of 
information.  From  this  perspective*  the  information  in  the 
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Figure  21.  Formulation  A:  Analysis  of  the  update  to  an  edge  of  probability 
q  between  two  parallel  lines,  one  of  probability  1  and  the 
other  of  probability  .5.  (a)  Neighborhood  of  central  edge 

and  vertex  probabilities.  (b)  Contributions  of  each  context 
Vij  to  the  probability  update  of  the  central  edge;  note  that 
the  central  edge  is  updated  to  at  least  .5,  independently  of 
its  initial  value. 
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surrounding  context  is  combined  in  a  multiplicative  way  with  the 
information  contained  in  the  central  edge  itself.  In  this 
formulation  it  is  difficult  for  either  piece  of  information  to 
completely  override  the  other.  The  growth  of  weak  edges  is 
slower  because  the  update  increments  are  smaller*  they  receive 
only  a  portion  of  the  support  available  from  the  context.  Strong 
edges*  on  the  other  hand*  for  which  there  is  evidence  of  support 
in  the  context*  grow  relatively  faster.  The  conjecture  is  that 
the  growth  of  edges  will  be  more  controlled  and*  where  necessary* 
slow  enough  to  allow  evidence  of  support  (or  non-support)  to 
propagate  from  “islands  of  reliability”  ELES773  to  local 
contexts. 


Thus*  Formulation  B  weights  the  contextual  evidence  by  the 
current  central  edge  probability  to  produce  updated  values 
P't+1<H): 

P,t+1(H-T)  -  PC(H-T)  *[J  l  Pt(H-TjV  )  Pt(V,,)] 

i  j  13  13 

and 

P,t+1(H«F)  .  pC(H-F)  *[£  l  PC(H-F|V  )  PC (V.  ) 

i  J  3  13 

For  clarity*  we  let 

f  -  I  J,  pc(h«t!v  )  Pt(V  ) 
i  j  13  13 


then  rewrite  the  equations  as 
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P'c+1(H-T)  -  PC(H-T)  *  f  (7) 

and  P’t+1(H-F)  -  [1-PC(H-T>]  *  [1-f] 

Clearly>  these  equations  are  not  wel  1-f ounded  in  probability 
theory  and  represent  a  different  heuristic  departure  from  the 
earlier  formulation.  In  particular*  since  the  updated  central 
edge  values  P't+l(H)  do  not  sum  to  1*  renormalization  is 
necessary  to  obtain  the  final  updated  values: 

Pt+1(H-T)  -  P,t+1(H«T)| [P,t+1(H-T)  +  Plt+1(H=F)] 

and 

Pt+1(H«F)  *  1  -  Pt+1(H=T) 

There  is  an  interesting  symmetry  in  the  update  equations 
<7).  The  current  central  edge  probability  and  the  contextual 
evidence  really  "weight"  each  other  equally.  Neither  is  favored 
by  the  formulation.  This  formulation  is  not  subject  to  the 
problem  noted  in  Figure  21  of  Formulation  A:  edges  between 
parallel  edges  are  updated  to  a  value  which  is  a  function  of  the 
probability  of  the  central  edge.  Figure  22  summarizes  the 
analysis  of  updating  two  parallel  lines*  one  of  probability 
1.  and  the  other  of  probability  .5*  one  pixel  apart.  Figure 
22(d)  shows  that  the  central  edge  update  in  this  case  is  very 
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dost  t~>  initial  probability  and  is  much  more  reasonable  than  the 
results  of  Figure  21. 

These  results  from  Formulation  B  are  encouraging  and 
application  to  natural  images  shows  the  algorithm  is  effective. 
Figure  3  and  Figures  23-26  summarize  various  aspects  of  these 
results.  Figure  23  shows  the  necessity  for  gradient  collection 
and  suppression;  23(a)  is  the  result  of  applying  Formulation  B 
in  the  absence  of  gradient  collection/suppression.  The  failure 
of  the  edge  process  is  most  obvious  in  the  roof  trim  and  bush 
boundary.  The  gradients  associated  with  the  boundaries  result  in 
a  series  of  "squares"  or  "rectangles"  connected  by  single  edges. 
Thus.  the  boundaries  consist  of  many  V22  vertices  and  as  the 
iterations  proceed,  the  edges  surrounded  by  the  V22  vertices 
eventually  disappear.  until  by  20  iterations  the  boundaries  are 
highly  fragmented.  Figure  23(b)  shows  the  same  edge  process 
applied  to  the  same  image  but  in  this  case  gradient  suppression 
was  included.  The  boundaries  are  quite  good,  although  not  all  of 
the  bush  boundary  and  shadow  boundary  have  been  found. 

The  V01  conditional  determines  the  amount  of  support  the 
central  edge  gets  from  the  context  when  continuation  of  the 
central  edge  occurs  to  one  side  only.  Variation  of  this 
parameter  has  the  effect  of  a  "sensitivity”  control  over  the 
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Figure  22.  Formulation  B:  Analysis  of  the  update  to  an  edge  of  probability 
q  between  two  parallel  lines,  one  of  probability  1.  and  the 
other  of  probability  .5.  (a)  Neighborhood  of  edge  of  probability 

q.  (b)  Contributions  of  each  context  Vij  to  the  probability 
update  of  the  central  edge,  (c)  Probability  of  central  edge 
at  next  iteration  expressed  as  a  function  of  q.  (d)  Table 
of  P(Rt+l)  as  a  function  of  q;  note  that  the  update  is  much 
more  reasonable  than  that  shown  in  Figure  21. 


continuation  edges  which  will  eventually  survive  into  tho  final 
boundary  imago*.  Th*  effect  is  shown  in  Figur*  24)  noto  that 
for  P<H|V01>  ■  .67#  tho  bush  boundary  is  complotod  but  othor 
"noisy"  binds  of  odgos  aro  also  ovidont#  notably  in  tho  troo  aroa 
to  tho  loft  of  tho  windows#  in  tho  shrubs  undor  th*  window#  and 
in  tho  right  sid*  of  th*  bush. 

Th*  loft  boundary  of  th*  bush#  particularly  whor*  it  cross** 
th*  window  trim  ucundary#  is  an  interesting  aroa  and  it 
highlights  son*  lingoring  problems  with  th*  edge  process#  as 
shown  in  Figur*  25.  Th*  intensity  close-up  in  25(b)  clearly 
shows  that  th*  placement  of  th*  bush  boundary  is  not  at  all 
obvious.  Th*  underlying  problem  with  this  area  is  that  a 
gradient  exists  between  th*  (white)  trim  and  th*  interior  of  th* 
bush  (dark).  Within  this  gradient  are  two  real  boundaries  on* 
pixel  apart:  the  boundary  between  the  trim  and  wall  and  th* 
boundary  between  the  wall  and  bush.  Furthermore#  th*  sign  of  th* 
contrast  of  both  boundaries  is  th*  same)  both  can  be  seen  in 
Figur*  25(d)*  which  represents  th*  initial  edge  probabilities. 
Figur*  25(*-h)  show  th*  results  after  1#  2*  5#  and  20  iterations* 
respectively.  Th*  wall/bush  boundary  is  suppressed  by  the 
wall/trim  boundary  because  it  is  a  much  weaker  boundary  and  is  of 
th*  same  direction.  One*  this  possible  connection  is  gone#  th* 


Results  from  Fqrmation  S  of  the  Edge  Process  Using 
the  VOl  Conditional  as  a  Boundary  Sensitivity 
Control.  (a)  Original  image.  <b)  P<H/V01)  =  .25. 

(  c  )  P  <  H/VOl )  »  .5.  <d>  P  (H/VOl )  =  .67.  All  results 
are  obtained  after  20  iterations  of  relaxation 
updating  using  the  conditionals  from  Figure  12<b) 
(except  for  VOl). 
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lure  25.  Results  from  Formulation  13  for  Bush/wal  1/trim 
Closeup.  (a)  Intensity  image  of  house  shouting 

location  of  bush/wal 1/trim  subimage.  <b)  8x9  blowup 
of  bush  area;  note  the  gradient  between 
tr im/wa 1 1 /bush  in  lower  left  quadrant.  (c)  Result  of 
1x2  mask;  very  weak  edger<  although  not  visible.  are 
present.  <d)  Gradient  collected  edge  probabilities. 

<  e-h  )  Iterations  1.  2>  5  and  i'O  (respectively)  of  the 

edge  relaxation  process.  The  junction  of  the  bush 
boundary  with  the  trim/walJ  represents  a  situation  in 
which  two  parallel  boundaries  of  the  same  sign  and 
one  pixel  apart  should  be  maintained/  but  are  not. 
The  final  results  arc-  still  a  reasonable 
approximation  to  the  boundary. 
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contest  of  the  oiiting  edge  in  Figurt  23(g)  switch**  primarily  to 
V21  and  tho  connection  edge  grows  in  (Figuro  23h>. 

Tho  loss  of  tho  wall/bush  boundary  is  inovitablo  given  tho 
conditions  in  tho  imago  (parallel  edges  of  same  sign*  one  pixel 
apart)  and  the  current  version  of  the  gradient 
collection/suppression  mechanisms.  These  mechanisms  assume  that 
any  non-zero  gradient  gives  rise  to  at  most  one  final  boundary. 
Tho  edge  process  then  attempts  to  find  the  best  placement  of  the 
individual  edges  making  up  the  boundary.  In  this  case  it  is 
doubtful  whether  any  purely  local  process  could  do  much  better. 

Figure  26  shows  results  from  Formulation  B  on  the  second 
house  image  (Figuro  lb).  Two  sequences  are  given*  one  for  VOl  - 
.23  and  one  for  VOl  ■  .  67i  within  each  sequence*  results  after 
1*  3*  and  10  iterations  are  shown.  Again*  the  results  are  quit* 
reasonable  given  the  starting  data.  For  the  first  time*  an 
undesirable  effect  of  local  normalization  over  the  llzll  window 
is  evident.  There  are  many  fairly  weak  edges  within  the  left 
wall  area  to  the  right  of  the  rightmost  large  window.  Edges 
within  the  influence  of  the  strong  boundaries  in  this  area  (e. g. » 
the  right  window  edges)  are  assigned  a  low  value  because  of  the 
doaiination  by  the  strong  boundary.  Edges  in  the  central  part  of 
this  area  only  see  other  weak  edges  and  hence  are  assigned  fairly 
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Fi aura  26.  Results  from  Formulation  U  for  Second  House  Image. 

(a)  Iterations  1,  5/  fuiti  10  for  VOl  =»  .25.  (b) 

Iterations  1>  5<  and  10  for  VOJ  =  .  67. 


high  values.  This  non-linear  mealing  gives  rise  to  the  vertical 
strip  of  edges  in  the  center  of  the  wall  which  can  be  seen  in  all 
the  images.  A  similar*  though  less  pronounced*  effect  can  be 
seen  in  the  roof. 

There  is  a  problem  in  extracting  continuous  boundaries  for 
the  dark  stripe  Mhich  forms  the  upper  border  of  the  roof.  The 
stripe  is  fragmented  into  pieces*  but  this  difficulty  appears  to 
be  related  to  the  coarse  resolution  rather  than  the  quality  of 
the  algorithm.  Any  very  narrow  diagonal  region  can  be  expected 
to  fragment. 

The  textural  lines  that  form  in  the  roof  in  26(b)  are 
reasonable  in  that  they  derive  from  the  visual  data.  Other 
processes  would  be  required  to  suppress  this  detail*  although 
some  of  that  effect  takes  place  by  running  the  algorithm  to 
extract  only  the  stronger  boundaries  as  in  Figure  26(a). 
However*  that  would  not  solve  the  problem  of  removing  lines  in  a 
strong  contrast  brick  wall.  We  leave  that  for  higher  levels  of 
organization  and  processing  CHAN78b3. 

Finally*  if  one  examines  results  in  close  detail*  there  are 
strange  effects  which  occur  in  some  cases  as  boundaries  approach 
each  other.  Other  related  problems  might  be  the  result  of 
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attempts  to  extend  incomplete  boundaries  after  the  initial  weak 
contextual  information  has  disappeared  during  relaxation.  There 
are  a  number  of  plausible  extensions  to  the  current  system  which 
would  provide  improved  edge  fidelity*  these  are  discussed  in  the 
conclusions. 
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V.  MEASURES  OF  PERFORMANCE 
V.  1  Fixed  Points*  Entroou,  ml  Consistency 

The  implication  of  our  assumption  that  the  conditionals 
remain  constant  is  that  any  fixed  point  in  the  relaxation  process 
is  a  state  in  which  all  edges  are  consistent  with  their  contexts* 
i. e.  *  all  fixed  points  are  consistent.  It  is  of  theoretical  and 
practical  interest  to  understand  which  states  are  fixed  points 
and  whether  any  given  set  of  initial  probabilities  will  converge 
to  a  fixed  point.  Rosenfeld  et.  al  EROS7&3  demonstrated  that 
iterative  linear  updating  functions  will  converge  to  a  fixed 
point  that  is  independent  of  the  initial  probabilities.  a 
characteristic  that  obviously  makes  that  form  of  processing 
useless.  One  solution  is  to  replace  the  linear  function  with  a 
non-linear  one.  although  this  requires  a  normalization  process  to 
restore  the  condition  that  Pt<H)  +  PC<H)  *»  i. 

The  updating  function*  defined  in  equation  6.  is  linear  in 
its  combination  of  the  conditional  edge  probability  with  its 
context.  However,  the  classical  relaxation  process  as  described 
above  treats  each  label  in  the  context  independently,  that  is 
each  of  the  labels  enters  into  a  linear  update  of  the  central 
In  the  process  described  here,  the  labels  in  the  context 
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label. 


are  jointly  used  in  the  update  function  via  the  vertex 
probabilities.  Viewed  from  the  vantage  of  the  individual  labels 
at  the  various  edge  locations  in  the  context*  this  is  a  highly 
non-linear  process. 

While  we  do  not  have  a  clear  theoretical  understanding  of 
the  properties  of  the  relaxation  process  described  here*  it  is 
obvious  that  there  are  a  very  large  number  of  fixed  points  given 
the  set  of  conditionals  already  defined.  Consider  any  array  of 
edge  probabilities  where  all  are  zero  except  for  closed  loops  of 
edges  with  probability  one.  The  only  edge  contexts  with  non-zero 
probability  are  V00.  V01  (for  edge  locations  adjacent  to  the 
probability  one  boundary)*  and  Vll.  The  updating  conditionals 
defined  in  Figure  12(b)  will  leave  each  edge  unchanged!  hence 
any  array  with  only  sets  of  closed  loops  of  certain  edges  is  a 
fixed  point. 

The  array  of  probabilities  has  zero  entropy  when  the 
probability  of  each  edge  in  the  array  is  O  or  1.  The  closed-loop 
boundary  examples  just  given  are  zero-entropy  arrays  which  are 
fixed  points  in  the  updating  process.  However*  not  all 
zero-entropy  arrays  are  fixed  points*  as  in  the  case  of  a  closed 
loop  of  probability  one  edges  with  one  edge  missing.  Conversely* 
there  are  fixed  points  which  do  not  have  zero-entropy.  It  is 
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also  easy  to  show  that  sets  of  conditionals  exist  which  produce 
oscillation  of  zero-entropy  arrays. 

Ona  can  viaw  tha  goal  of  tha  updating  procass  to  be  tha 
transformation  of  an  array  of  adga  lihalihoods  with  non-zero 
antropy  into  a  zero-entropy  fizad  point  which  is  by  soma  maasura 
"closest"  to  tha  initial  valuas.  There  are  a  variaty  of  possibla 
maasuras  of  closeness,  such  as  tha  maan  square  diffarenca  batwaen 
tha  initial  and  final  arrays. 


V.  2  ftlflftJLl  Bnt.nr.tt  2±  Bui  ft,.  *M  Inc onsl stone u 

Ona  of  tha  kay  concerns  of  those  interested  in  relaxation 
labelling  algorithms  is  tha  lack  of  any  criteria  for  global 
optimization.  Tha  problem  of  finding  the  minimum  distance  fixed 
point  with  zero  entropy  could  be  formulated  as  a  linear 
programming  problem  on  tha  sat  of  edge  values,  but  it  does  not 
appear  computationally  feasible  when  the  number  of  such  edges  is 
of  the  order  of  500,000.  Instead  we  have  chosen  parallel 
mechanisms  for  organizing  local  contexts  in  a  goal-or ientad  way 
in  order  to  achieve  a  global  organization  of  this  information. 
This  heuristic  approach  allows  efficient  local  processing  at  the 
risk  of  producing  a  non-optimal  global  result.  Therefore,  it  is 
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useful  to  have  empirical  ntaturti  of  the  global  performance  of 
the  iterative  updating  process  over  time.  The  following  measures 
defined  across  the  14  edges  in  our  array  will  hopefully  lend  some 
empirical  understanding  of  the  processing. 

The  first  measure*  entropy*  characterizes  the  degree  of 

ambiguity  or  uncertainty  in  the  edge  array: 

U 

Qi  *  “ti  Pt(H.)logPt(H)] 
i-1  1 

This  measure  is  zero  only  when  each  edge  in  the  array  has 
probability  zero  or  one.  Although  minimization  of  entropy  is 
desirable  in  most  cases*  one  must  remember  that  most  of  the  huge 
number  of  possible  global  zero-entropy  cases  will  entirely 
violate  the  desired  relationships  embodied  in  the  conditionals. 

A  more  important  measure  is  inconsistency: 

w 

Qo  "  l  [Pt(H1)  ”  Pt-1(H.)J**2 
*  i-1  1 

This  measure  is  the  only  one  related  to  the  conditionals  which 
embodies  the  semantics  of  the  domain.  It  is  a  function  of  the 
degree  to  which  likelihoods  of  local  edges  are  inconsistent  with 
their  context^  Due  to  the  nature  of  our  updating  formula*  this 
measure  follows  directly  from  the  change  in  the  distribution  of 
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likelihoods  from  one  iteration  to  the  next.  Consequently*  it  is 
a  aeasure  of  stability  of  the  relaxation  process*  since  it  will 
reflect  convergence  of  the  distributions  to  a  fixed  point*  as 
well  as  oscillatory  behavior. 

Finally  we  define  "drift1**  a  measure  of  the  divergence  from 

the  initial  distribution  of  edge  probabilities: 
r  W 

Q  \m  l  [PCOU  -  P°(H.)]**2 
i-1  1  1 

While  one  may  choose  the  goal  to  be  a  set  of  likelihoods  which  is 
consistent  throughout  the  network*  the  result  should  be  a 
function  of  the  initial  data.  It  is  desirable  that  the  final 
result  be  close  “almost  everywhere"  to  the  initial  probabilities 
from  which  the  updating  has  been  driven. 

Each  of  the  three  measures  appears  to  focus  upon  a  different 
aspect  of  the  relaxation  labelling  process.  Ideally*  one  would 
like  the  system  to  converge  to  a  fixed  point  which  simul taneously 
minimixes  some  function  of  entropy  and  drift.  The  manner  in 
which  this  can  be  globally  achieved  via  local  updating  processes 

*J  * 

is  an  open  question*  and  in  fact  may  only  be  possible  in 
restricted  situations. 
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VI.  CONCLUSION 

The  theoretical  Bayesian  formulation  which  is  valid  for  a 
single  edge  and  its  context  was  expanded  into  a  heuristic  process 
for  iteratively  updating  in  parallel  the  array  of  edge 
probabilities.  Unfortunately*  experimental  results  demonstrated 
the  ineffectiveness  of  this  formulation.  A  variety  of  heuristic 
modifications*  often  suggested  by  theoretical  considerations* 
were  explored*  finally  resulting  in  an  algorithm  that  performs  in 
an  effective  manner  on  a  number  of  very  complex  images.  It  is 
interesting  that  the  process  effectiveness  was  dependent  upon  a 
process  of  normalization  which  introduced  competition  between 
alternative  labels. 


VI.  1  Thtgrffti&il 


It  is  worthy  of  note  that  in  all  algorithms  explored  the 
values  of  the  set  of  conditional  probabilities  that  were  derived 
from  the  model  of  desired  line  drawings  has  remained  basically 
fixed.  It  has  not  been  necessary  to  "tune"  a  set  of  weights  for 
different  images*  or  even  for  application  of  the  algorithm  to  the 
first  image.  The  theoretical  estimates  have  worked  very  well 
once  an  effective  form  of  the  algorithm  was  developed*  and  at  the 
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s ame  tint*  results  were  consistently  bed  for  ell  verietions  in  the 
parameters  with  the  ineffective  algorithms.  Variation  of  one 
conditional  probability  in  the  final  algorithm  provides  a 
mechanism  for  varying  the  contrast  sensitivity  of  the  boundaries 
extracted. 


VI.  2  Model  fat  Qra dient  Bflmulam 


Another  key  aspect  of  our  edge-boundary  algorithm  is  the  use 
of  a  model  for  the  gradient  of  boundaries.  The  representative 
strength  and  location  of  an  edge  must  be  derived  from  a  wide 
non-zero  gradient  across  the  feature  values  of  pixels.  Such 
gradient  boundaries  are  typical  in  images  and  in  many  cases 
results  which  match  human  perception  and  intuition  are  dependent 
upon  the  extraction  of  the  total  contrast  and  placement  of  that 
edge  strength  within  the  gradient  width.  Further  studies  have 
been  initiated  to  improve  this  model  by  using  factors  of  boundary 
contrast!  boundary  width#  and  values  of  the  first  and  second 
derivatives  of  the  changes  in  the  feature  values. 
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vi.  3  ui£  fif  jflit nUr\i  bLLiM 

Th »  algorithm  presented  in  this  paper  can  be  extended  by 
employing  additional  local  contextual  information  beyond  the 
joint  probabilities  which  reflect  the  likelihood  of  edge  presence 
in  the  context.  Local  support  via  boundary  continuity  can  be 
dependent  upon  the  continuing  edges  having  similar 
characteristics  of  edge  width*  with  support  decoupled  to  the 
degree  that  edge  width  of  a  pair  of  edges  differs.  This  avoids 
the  problem  of  labelling  edge  types*  yet  still  takes  advantage  of 
edge  widths  without  violation  of  Harr's  principle  of  least 
commitment!  it  just  requires  the  parameterixation  of  width  as  a 
feature  instead  of  classification  into  a  small  number  of  types  of 
edge  widths  CMARR763. 

It  may  be  possible  to  take  full  advantage  of  all  the 
information  in  a  gradient*  by  integrating  the  boundary  gradient 
model  more  fully  into  the  updating  process.  Ehrich  CEHR79b3  and 
Quam  CQUA78D  have  both  employed  measures  of  similarity  of  the 
intensity  profiles  of  adjacent  scan  lines  in  the  grouping  of 
edges  into  boundaries. 


VI.  4  miiila  a±  Initial  Sensoru  Data 


There  ara  other  potentially  useful  mechanitnt  by  which  tha 
local  context  can  serve  to  furthar  guide  tha  boundary  formation 
process.  Tha  characteristic s  of  adjacent  pixels  to  either  aide 
of  a  pair  of  edgaa  can  be  compared  ao  that  similar  region 
properties  will  land  furthar  support.  In  this  way  tha  initial 
sensory  data  maintains  an  influence  throughout  tha  updating 
process. 

Another  problem  is  that  by  tha  time  that  contextual 

information  propagates  in  from  surrounding  and  lass  ambiguous 
areas*  it  is  quite  possible  that  ambiguous  data  will  have 
disappeared  during  relaxation  updating*  leaving  no  trace  of  the 
original  information.  Tha  initial  sensory  data  can  be  used  to 
alleviate  this  problem  by  restarting  the  iterative  updating 
process  after  N  iterations  with  a  value  at  each  edge  H  of 
MAX<PO<H)*  PN<H>).  The  effect  is  to  allow  boundaries*  which 
have  been  organized  by  N  iterations  of  relaxation  updating*  to 
influence  the  further  organization  of  ambiguous  edges  without  the 
problem  of  properly  timing  the  propagation  of  this  contextual 
information.  Some  initial  experiments  in  this  direction  have 
shown  this  technique  to  be  promising. 


VI.  5  Variation  a£  Contrast  Sensitivitu 

An  important  aspect  of  the  algorithm  is  the  ability  to  vary 
the  sensitivity  of  the  edge  contrast  in  forming  boundaries.  For 
example*  the  boundary  of  an  object  in  the  image  may  lie  partially 
in  shade  causing  reduced  contrast  between  object  and  background* 
yet  the  continuity  and  consistency  of  that  boundary  make  the 
boundary  obvious.  The  algorithm  must  vary  its  sensitivity  to 
edge  strength.  The  conditional  probability  P<H*T/V01)  has  been 
used  to  extend  boundary  terminations  in  an  effort  to  extract  less 
certain  boundaries*  and  has  yielded  encouraging  results.  An 
alternative  which  ought  to  be  explored  is  variation  of  the 
initial  probabilities  accomplished  by  means  of  changes  in  the 
value  of  K  in  equation  1.  Reducing  K  has  the  effect  of 
increasing  the  initial  edge  probabilities. 


VI- 6  Ijrqyr  Upcal  C.ontpxt 

Finally*  the  use  of  a  local  larger  context  than  the 
eight-edge  context  developed  here  would  allow  more  effective 
patterns  of  boundary  continuity  to  be  employed.  There  would  be 
more  extensive  overlap  of  adjacent  local  contexts*  thereby  giving 
a  better  perspective  of  the  relationship  between  contiguous  edges 
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in  the  consistent  organization  of  continuous  boundaries.  The 
computational  overhead»  however*  increases  exponentially  with  the 
number  of  edges  in  the  context  and  may  not  be  feasible  even  for  a 
context  of  edges  of  length  two  to  either  side.  This  would 
involve  conditionals  based  upon  22  edges  around  the  central  edge* 
which  is  already  beyond  consideration  via  the  approach  developed 
in  this  paper. 
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